# kalpeshgamit/codebase-pilot architecture

> Architecture diagram of the GitHub repository kalpeshgamit/codebase-pilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kalpeshgamit/codebase-pilot
- Repository: https://github.com/kalpeshgamit/codebase-pilot

## Overview

1. **What this repo is**
   - `codebase-pilot` is a local-first Node.js/TypeScript toolchain for preparing codebases as AI context. It does not call AI APIs; instead it scans a repository, scores and compresses it, and exposes the result through CLI, MCP, Claude Code plugin assets, and a web dashboard.

2. **Primary entry points**
   - The main executable is `src/bin/codebase-pilot.ts`, which fans out into command modules under `src/cli/`.
   - The same core capabilities are also surfaced through `src/mcp/server.ts` for MCP clients and `src/ui/server.ts` for the dashboard.
   - The `plugin/` tree packages Claude Code plugin metadata, hooks, and skills so the same engine can be driven from inside an AI IDE workflow.

3. **Core pipeline**
   - The central data flow is: **scan → detect → pack → compress → format → count/record**.
   - `src/scanner/` discovers repository structure, languages, frameworks, databases, test runners, and existing project config.
   - `src/packer/` collects selected files, applies token accounting, and formats output as Markdown or XML.
   - `src/compress/` reduces prompt size by preserving signatures and structure while folding implementation bodies.

4. **Repository intelligence layer**
   - `src/intelligence/` is the decision-making layer that powers “what matters” selection.
   - It includes incremental change detection, git-derived signals, import/dependency analysis, task selection, and graph visualization support.
   - This layer is what enables commands like affected-only packing, prune-by-import-graph, blast-radius analysis, and compare-diff token impact.

5. **Security boundary**
   - `src/security/` provides a standalone secret scanner using a large pattern set.
   - Security scanning is both a first-class CLI operation and an automatic gate in the packing flow: files containing secrets are excluded from packed output.
   - This is an important architectural boundary because it separates “context optimization” from “do not leak credentials.”

6. **Registries and detection catalogs**
   - `src/registry/` stores curated knowledge about languages, frameworks, databases, and test tools.
   - These registries are used by the scanner to classify projects and by generators to produce project-specific configuration.
   - In diagrams, this is the metadata/catalog subsystem feeding detection and setup.

7. **Local persistence and search**
   - `src/memory/db.ts` and `src/scanner/database.ts` indicate local database-backed storage for usage history and indexing.
   - The README also describes full-text search with ranking and snippets, so there is a local searchable data layer behind commands like `search`, `stats`, `compare`, and dashboard views.
   - The architecture is designed around local state accumulation over time, not remote services.

8. **Web dashboard and daemon**
   - `src/ui/` is a separate runtime path for the always-on dashboard and background service.
   - `src/ui/server.ts`, `ws.ts`, and `pack-worker.ts` suggest a Node server serving pages, pushing real-time updates over WebSockets, and offloading packing work to a worker process.
   - `src/ui/daemon.ts` plus the `service` command indicate a long-lived local daemon model that records usage continuously.

9. **MCP and agent integration**
   - `src/mcp/server.ts` exposes the engine as MCP tools and prompts, so other AI clients can consume scanning, packing, token counting, health checks, and file listing.
   - `src/agents/generator.ts` plus `plugin/skills/codebase-pilot/` represent the agent orchestration layer used by Claude Code workflows.
   - The boundary here is: the core engine stays reusable, while client-specific integration is layered on top.

10. **Project configuration generation**
   - `src/generators/` creates repo-local AI tooling artifacts such as `CLAUDE.md`, `agents.json`, ignore files, slash-command definitions, and platform rule files.
   - This subsystem turns scan results into installed project configuration, which is why `init` is a high-level orchestration command.
   - It also validates output before writing, so generation is treated as a controlled boundary rather than ad hoc file edits.

11. **CLI command families**
   - The CLI is organized around distinct operational modes: initialization, scanning, packing, token analysis, impact analysis, security scanning, visualization, dashboard control, service management, and repair/eject utilities.
   - `compare`, `impact`, `tokens`, `eval`, `health`, `fix`, and `eject` are not separate engines; they are views and maintenance actions over the same scanned/packed local data.

12. **Distribution and integration surface**
   - The repo also ships deployment/integration assets: GitHub Actions, Homebrew formula, install script, and Claude Code marketplace metadata.
   - These do not change the core architecture, but they show that the product is meant to be installed as a CLI, embedded as a plugin, run as a daemon, or invoked in CI.

13. **Repository-level architecture summary**
   - The main boundary is between **core analysis engines** (`scanner`, `intelligence`, `compress`, `packer`, `security`, `registry`) and **presentation/integration layers** (`cli`, `ui`, `mcp`, `plugin`, `generators`).
   - Data typically flows from repository contents into scan/index state, then into packed/compressed AI context, and finally into local persistence and UI/MCP outputs.
   - Everything is intentionally local and offline, with no external cloud dependency in the runtime path.

Main entry points: [`src/packer/index.ts`](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/packer/index.ts) (Packer), [`src/cli/pack.ts`](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/cli/pack.ts) (Commands), [`src/scanner/detector.ts`](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/scanner/detector.ts) (Scanner), [`src/bin/codebase-pilot.ts`](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/bin/codebase-pilot.ts) (Main CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core engine"]
  node_scanner["Scanner<br/>analysis engine<br/>[detector.ts]"]
  node_registry["Catalogs<br/>metadata registry<br/>[index.ts]"]
  node_intel["Intelligence<br/>decision layer<br/>[incremental.ts]"]
  node_compress["Compressor<br/>context reducer<br/>[index.ts]"]
  node_packer["Packer<br/>context builder<br/>[index.ts]"]
  node_security["Secret scanner<br/>security boundary<br/>[scanner.ts]"]
end

subgraph group_interfaces["Interfaces"]
  node_bin["Main CLI<br/>entrypoint<br/>[codebase-pilot.ts]"]
  node_cli["Commands<br/>cli modules<br/>[pack.ts]"]
  node_mcp["MCP server<br/>tool server<br/>[server.ts]"]
  node_ui["Dashboard<br/>web runtime<br/>[server.ts]"]
  node_plugin["Claude plugin<br/>integration bundle<br/>[plugin.json]"]
end

subgraph group_generation["Generation"]
  node_generators["Generators<br/>artifact writers<br/>[validate.ts]"]
  node_agentgen["Agent generator<br/>orchestration helper<br/>[generator.ts]"]
end

subgraph group_storage["Local state"]
  node_memory[("Memory DB<br/>local persistence<br/>[db.ts]")]
  node_scan_db[("Scan index<br/>local index<br/>[database.ts]")]
end

subgraph group_assets["Distribution"]
  node_formula["Formula<br/>distribution asset<br/>[codebase-pilot.rb]"]
  node_workflow["CI workflows<br/>automation asset<br/>[ci.yml]"]
  node_marketplace["Marketplace<br/>distribution asset<br/>[marketplace.json]"]
end

node_root["Core types<br/>shared types<br/>[types.ts]"]
node_index["Library API<br/>package entry<br/>[index.ts]"]

node_bin -->|"dispatches"| node_cli
node_cli -->|"scan"| node_scanner
node_cli -->|"pack"| node_packer
node_cli -->|"analyze"| node_intel
node_cli -->|"scan secrets"| node_security
node_cli -->|"serve"| node_ui
node_ui -->|"records"| node_memory
node_ui -->|"worker packs"| node_packer
node_mcp -->|"tools"| node_scanner
node_mcp -->|"tools"| node_packer
node_plugin -->|"drives"| node_agentgen
node_scanner -->|"classifies"| node_registry
node_scanner -->|"persists"| node_scan_db
node_scanner -->|"signals"| node_intel
node_intel -->|"selects"| node_packer
node_security -->|"gates"| node_packer
node_compress -->|"reduces"| node_packer
node_packer -->|"logs"| node_memory
node_generators -->|"uses"| node_registry
node_scanner -->|"feeds"| node_generators
node_agentgen -->|"writes"| node_plugin
node_index -->|"exports"| node_bin
node_workflow -->|"builds"| node_bin
node_formula -->|"packages"| node_bin
node_marketplace -->|"publishes"| node_plugin

click node_bin "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/bin/codebase-pilot.ts"
click node_cli "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/cli/pack.ts"
click node_mcp "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/mcp/server.ts"
click node_ui "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/ui/server.ts"
click node_plugin "https://github.com/kalpeshgamit/codebase-pilot/blob/main/plugin/.claude-plugin/plugin.json"
click node_scanner "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/scanner/detector.ts"
click node_registry "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/registry/index.ts"
click node_intel "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/intelligence/incremental.ts"
click node_compress "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/compress/index.ts"
click node_packer "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/packer/index.ts"
click node_security "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/security/scanner.ts"
click node_memory "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/memory/db.ts"
click node_scan_db "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/scanner/database.ts"
click node_generators "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/generators/validate.ts"
click node_agentgen "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/agents/generator.ts"
click node_root "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/types.ts"
click node_index "https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/index.ts"
click node_formula "https://github.com/kalpeshgamit/codebase-pilot/blob/main/Formula/codebase-pilot.rb"
click node_workflow "https://github.com/kalpeshgamit/codebase-pilot/blob/main/.github/workflows/ci.yml"
click node_marketplace "https://github.com/kalpeshgamit/codebase-pilot/blob/main/.claude-plugin/marketplace.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scanner,node_registry,node_intel,node_compress,node_packer,node_security toneBlue
class node_bin,node_cli,node_mcp,node_ui,node_plugin toneAmber
class node_generators,node_agentgen toneMint
class node_memory,node_scan_db toneRose
class node_formula,node_workflow,node_marketplace toneIndigo
class node_root,node_index toneNeutral
```

## Components

### Core engine

Local analysis, packing, compression, security, and metadata subsystems.

- [Scanner](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/scanner/detector.ts): analysis engine · Discovers structure, languages, frameworks, databases, tests, and existing config. · `src/scanner/detector.ts`
- [Catalogs](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/registry/index.ts): metadata registry · Curated language, framework, database, and testing knowledge. · `src/registry/index.ts`
- [Intelligence](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/intelligence/incremental.ts): decision layer · Incremental signals, import analysis, task selection, search, and visualization helpers. · `src/intelligence/incremental.ts`
- [Compressor](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/compress/index.ts): context reducer · Shrinks code context while preserving signatures and structure. · `src/compress/index.ts`
- [Packer](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/packer/index.ts): context builder · Collects selected files, counts tokens, and formats output. · `src/packer/index.ts`
- [Secret scanner](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/security/scanner.ts): security boundary · Pattern-based secret detection that gates packed output. · `src/security/scanner.ts`

### Interfaces

CLI, MCP, UI, and plugin entry points that expose the engine.

- [Main CLI](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/bin/codebase-pilot.ts): entrypoint · Primary executable that dispatches command families. · `src/bin/codebase-pilot.ts`
- [Commands](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/cli/pack.ts): cli modules · Operational command handlers for scan, pack, compare, health, and maintenance. · `src/cli/pack.ts`
- [MCP server](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/mcp/server.ts): tool server · Exposes scanning, packing, token, and health capabilities to MCP clients. · `src/mcp/server.ts`
- [Dashboard](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/ui/server.ts): web runtime · Local server, WebSocket updates, and worker-backed UI runtime. · `src/ui/server.ts`
- [Claude plugin](https://github.com/kalpeshgamit/codebase-pilot/blob/main/plugin/.claude-plugin/plugin.json): integration bundle · Claude Code plugin metadata, hooks, and skills package. · `plugin/.claude-plugin/plugin.json`

### Generation

Project configuration and agent artifact writers.

- [Generators](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/generators/validate.ts): artifact writers · Produces Claude, agent, ignore, slash-command, and platform rule files. · `src/generators/validate.ts`
- [Agent generator](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/agents/generator.ts): orchestration helper · Builds agent-facing configuration and workflow assets. · `src/agents/generator.ts`

### Local state

Persistent memory, indexing, and searchable local data.

- [Memory DB](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/memory/db.ts): local persistence · Local usage history and searchable state backing commands and dashboard views. · `src/memory/db.ts`
- [Scan index](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/scanner/database.ts): local index · Repository scanning database used for persistent discovery and retrieval. · `src/scanner/database.ts`

### Distribution

Packaging and integration assets for install and marketplace use.

- [Formula](https://github.com/kalpeshgamit/codebase-pilot/blob/main/Formula/codebase-pilot.rb): distribution asset · Homebrew packaging definition for installation. · `Formula/codebase-pilot.rb`
- [CI workflows](https://github.com/kalpeshgamit/codebase-pilot/blob/main/.github/workflows/ci.yml): automation asset · GitHub Actions workflows for CI and token reporting. · `.github/workflows/ci.yml`
- [Marketplace](https://github.com/kalpeshgamit/codebase-pilot/blob/main/.claude-plugin/marketplace.json): distribution asset · Plugin marketplace metadata for Claude Code distribution. · `.claude-plugin/marketplace.json`

### Other components

- [Core types](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/types.ts): shared types · Common shared types and utility glue used across subsystems. · `src/types.ts`
- [Library API](https://github.com/kalpeshgamit/codebase-pilot/blob/main/src/index.ts): package entry · Top-level module export for programmatic access. · `src/index.ts`

## Connections

- Main CLI → Commands: dispatches · Main executable fans out into command modules.
- Commands → Scanner: scan · Scan commands invoke repository discovery.
- Commands → Packer: pack · Pack commands build compressed AI context.
- Commands → Intelligence: analyze · Impact, compare, and task-aware commands use intelligence signals.
- Commands → Secret scanner: scan secrets · Security scanning is exposed as a first-class command.
- Commands → Dashboard: serve · CLI can launch or manage the dashboard runtime.
- Dashboard → Memory DB: records · Dashboard and daemon persist usage history locally.
- Dashboard → Packer: worker packs · Worker process offloads packing work.
- MCP server → Scanner: tools · MCP clients can trigger scanning-related operations.
- MCP server → Packer: tools · MCP clients can request packing and token counting.
- Claude plugin → Agent generator: drives · Plugin workflow consumes generated agent assets.
- Scanner → Catalogs: classifies · Scanner uses catalogs to identify repo technologies.
- Scanner → Scan index: persists · Scan results and repository state accumulate locally.
- Scanner → Intelligence: signals · Discovery outputs feed intelligence features.
- Intelligence → Packer: selects · Intelligence chooses what should be packed.
- Secret scanner → Packer: gates · Secret detection excludes sensitive files from output.
- Compressor → Packer: reduces · Compression shrinks file bodies before formatting.
- Packer → Memory DB: logs · Token and usage logging update local state.
- Generators → Catalogs: uses · Generators rely on repo catalogs to produce tailored config.
- Scanner → Generators: feeds · Scan results are transformed into project-local artifacts.
- Agent generator → Claude plugin: writes · Agent generation supports plugin and Claude workflow assets.
- Library API → Main CLI: exports · Library entrypoint ties together the executable surface.
- CI workflows → Main CLI: builds · Automation validates and ships the CLI/tooling.
- Formula → Main CLI: packages · Homebrew formula packages the command-line distribution.
- Marketplace → Claude plugin: publishes · Marketplace metadata publishes the plugin bundle.