# mathews-tom/archex architecture

> Architecture diagram of the GitHub repository mathews-tom/archex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mathews-tom/archex
- Repository: https://github.com/mathews-tom/archex

## Overview

1. **What the system is**
   archex is a local repository-context engine for agents. It does not answer questions itself; it builds a ranked, token-budgeted context bundle plus a receipt that explains freshness, completeness, skipped candidates, and omitted edges.

2. **Primary entry surfaces**
   The same core capabilities are exposed through several front doors: the CLI under `src/archex/cli/`, the Python API in `src/archex/api.py`, the MCP server in `src/archex/integrations/mcp.py`, and Docker images built from `docker/`. These are delivery surfaces, not separate implementations.

3. **Repo-local state boundary**
   All generated state lives under `.archex/` in the target repository. That boundary holds settings, metadata, SQLite index data, optional vector artifacts, graph artifacts, and dogfood/baseline files. The project is designed to run locally by default with no hosted dependency.

4. **Repository acquisition**
   Input repositories are discovered and normalized by `src/archex/acquire/`:
   - `local.py` for the current checkout
   - `git.py` for git-based sources
   - `discovery.py` for repo detection and setup  
   This is the ingestion boundary between a filesystem/repo and the rest of the pipeline.

5. **Language parsing and structural extraction**
   `src/archex/parse/` is the structural extraction layer. It contains language adapters for full-tier languages, structured markup/config languages, and chunk-only languages. The parser extracts symbols, imports, and cross-file references where the tier supports them, and falls back to chunking for lower-fidelity formats.

6. **Analysis layer**
   `src/archex/analyze/` turns parsed structure into higher-level signals:
   - `interfaces.py` for interface and type relationships
   - `modules.py` for module-level structure
   - `patterns.py` for architectural pattern detection
   - `decisions.py` for inferred design decisions  
   This layer feeds both retrieval quality and architecture-oriented commands.

7. **Indexing and retrieval core**
   `src/archex/index/` is the main retrieval subsystem. It combines:
   - BM25-style lexical search
   - optional vector retrieval and embeddings
   - sparse/rerank components
   - graph-based relationships
   - artifact/store management and delta handling  
   This is where candidate files, symbols, and chunks are scored and merged into a final ranking.

8. **Query pipeline and bundle assembly**
   `src/archex/pipeline/` and top-level query-related modules such as `graph_query.py`, `explain.py`, `scout.py`, and `impact.py` assemble the final response. The pipeline expands the initial retrieval set with graph/type context, applies token budgets, and emits the bundle the agent will actually consume.

9. **Serving and rendering the context**
   `src/archex/serve/` is the prompt-facing presentation layer. It handles context packing, modality/intent handling, compression, and output rendering to JSON, Markdown, or XML. This is the final transformation from ranked internal evidence into a consumable context envelope.

10. **Receipts, status, and trust signals**
   `src/archex/receipt.py`, `status.py`, `doctor.py`, and related modules define the operational trust surface. Receipts explain what was included or skipped; doctor/status report index health, grammar availability, and freshness; these outputs are central to the repository’s “safe-to-act” story.

11. **Metrics and privacy boundary**
   `src/archex/metrics/` is a separate, opt-in local telemetry substitute: it records anonymous usage counters on the machine only, with no source code, query text, or prompt bodies in event rows. It is intentionally isolated from the core retrieval flow and from any hosted upload path.

12. **Benchmarking and regression control**
   `src/archex/benchmark/` plus the checked-in `benchmarks/` corpus form a full evaluation harness. It drives task suites, baseline comparisons, gate checks, head-to-head reports, delta runs, and bundle-only evaluator flows. This is the main quality-control loop for retrieval, ranking, compression, and language promotion decisions.

13. **Integrations and extension points**
   `src/archex/integrations/` contains optional adapters for LangChain, LlamaIndex, LSAP/LSP-style enrichment, and MCP. The parser and language layers are also extensible via adapter registration, which is how new languages enter the system.

14. **Tests and fixtures as architecture evidence**
   The large `tests/` tree and `tests/fixtures/` corpus are not just verification; they encode the intended boundaries of the system. The fixtures cover many languages and repository shapes, mirroring the parser tiers, graph behavior, benchmark tasks, and CLI lifecycle commands.

15. **End-to-end flow to diagram**
   The main architecture flow is: repository source -> acquisition -> parse/analyze -> index/store in `.archex/` -> retrieval/ranking/graph expansion -> budgeted bundle + receipt -> CLI/API/MCP/Docker consumer. The benchmark and metrics subsystems sit alongside this flow as validation and observability, not as part of the primary retrieval path.

Main entry points: [`src/archex/serve`](https://github.com/mathews-tom/archex/tree/main/src/archex/serve) (Serve), [`src/archex/index`](https://github.com/mathews-tom/archex/tree/main/src/archex/index) (Index), [`src/archex/pipeline`](https://github.com/mathews-tom/archex/tree/main/src/archex/pipeline) (Pipeline), [`src/archex/benchmark`](https://github.com/mathews-tom/archex/tree/main/src/archex/benchmark) (Benchmark)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_core["Core engine"]
  node_repo_source[("Repo source<br/>repository input")]
  node_acquire_layer["Acquire<br/>ingestion"]
  node_parse_layer["Parse<br/>structural extract"]
  node_analyze_layer["Analyze<br/>semantic analysis"]
  node_index_layer["Index<br/>retrieval core"]
  node_pipeline_layer["Pipeline<br/>query assembly"]
  node_serve_layer["Serve<br/>context packing"]
  node_receipt["Receipt<br/>trust report<br/>[receipt.py]"]
  node_graph_query{{"Graph query<br/>context expansion<br/>[graph_query.py]"}}
  node_languages["Languages<br/>adapter registry<br/>[languages.py]"]
end

subgraph group_group_surface["Delivery surfaces"]
  node_cli["CLI<br/>command surface"]
  node_api["Python API<br/>library surface<br/>[api.py]"]
  node_mcp["MCP server<br/>integration surface<br/>[mcp.py]"]
  node_docker["Docker<br/>packaging surface"]
end

subgraph group_group_validation["Validation and observability"]
  node_metrics["Metrics<br/>local telemetry"]
  node_benchmark["Benchmark<br/>evaluation harness"]
  node_doctor_status["Doctor/status<br/>health checks<br/>[doctor.py]"]
end

subgraph group_group_state["Local state"]
  node_state_store[(".archex<br/>repo-local state<br/>[.archex]")]
end

node_repo_source -->|"discover"| node_acquire_layer
node_acquire_layer -->|"parse"| node_parse_layer
node_parse_layer -->|"derive signals"| node_analyze_layer
node_analyze_layer -->|"index"| node_index_layer
node_index_layer -->|"expand"| node_graph_query
node_index_layer -->|"ranked candidates"| node_pipeline_layer
node_graph_query -->|"context enrich"| node_pipeline_layer
node_pipeline_layer -->|"bundle"| node_serve_layer
node_serve_layer -->|"emit receipt"| node_receipt
node_pipeline_layer -.->|"track omissions"| node_receipt
node_serve_layer -->|"render"| node_cli
node_serve_layer -->|"serve"| node_api
node_serve_layer -->|"serve"| node_mcp
node_serve_layer -->|"package"| node_docker
node_index_layer -->|"persist"| node_state_store
node_metrics -->|"report"| node_doctor_status
node_benchmark -.->|"evaluate"| node_index_layer
node_benchmark -.->|"evaluate"| node_pipeline_layer
node_benchmark -.->|"gate"| node_doctor_status
node_languages -->|"register adapters"| node_parse_layer
node_doctor_status -->|"surface"| node_cli

click node_acquire_layer "https://github.com/mathews-tom/archex/tree/main/src/archex/acquire"
click node_parse_layer "https://github.com/mathews-tom/archex/tree/main/src/archex/parse"
click node_analyze_layer "https://github.com/mathews-tom/archex/tree/main/src/archex/analyze"
click node_index_layer "https://github.com/mathews-tom/archex/tree/main/src/archex/index"
click node_pipeline_layer "https://github.com/mathews-tom/archex/tree/main/src/archex/pipeline"
click node_serve_layer "https://github.com/mathews-tom/archex/tree/main/src/archex/serve"
click node_receipt "https://github.com/mathews-tom/archex/blob/main/src/archex/receipt.py"
click node_state_store "https://github.com/mathews-tom/archex/blob/main/.archex"
click node_cli "https://github.com/mathews-tom/archex/tree/main/src/archex/cli"
click node_api "https://github.com/mathews-tom/archex/blob/main/src/archex/api.py"
click node_mcp "https://github.com/mathews-tom/archex/blob/main/src/archex/integrations/mcp.py"
click node_docker "https://github.com/mathews-tom/archex/tree/main/docker"
click node_metrics "https://github.com/mathews-tom/archex/tree/main/src/archex/metrics"
click node_benchmark "https://github.com/mathews-tom/archex/tree/main/src/archex/benchmark"
click node_doctor_status "https://github.com/mathews-tom/archex/blob/main/src/archex/doctor.py"
click node_graph_query "https://github.com/mathews-tom/archex/blob/main/src/archex/graph_query.py"
click node_languages "https://github.com/mathews-tom/archex/blob/main/src/archex/languages.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_repo_source,node_acquire_layer,node_parse_layer,node_analyze_layer,node_index_layer,node_pipeline_layer,node_serve_layer,node_receipt,node_graph_query,node_languages toneBlue
class node_cli,node_api,node_mcp,node_docker toneAmber
class node_metrics,node_benchmark,node_doctor_status toneMint
class node_state_store toneRose
```

## Components

### Core engine

Primary local repository-context pipeline.

- **Repo source**: repository input · Target repository discovered from the local filesystem or git source.
- [Acquire](https://github.com/mathews-tom/archex/tree/main/src/archex/acquire): ingestion · Normalizes local, git, and discovery inputs into a usable repo context. · `src/archex/acquire`
- [Parse](https://github.com/mathews-tom/archex/tree/main/src/archex/parse): structural extract · Language adapters and parsing engine for symbols, imports, and chunking. · `src/archex/parse`
- [Analyze](https://github.com/mathews-tom/archex/tree/main/src/archex/analyze): semantic analysis · Derives interfaces, module structure, patterns, and design decisions. · `src/archex/analyze`
- [Index](https://github.com/mathews-tom/archex/tree/main/src/archex/index): retrieval core · BM25, vector, graph, fusion, rerank, artifact, and store management. · `src/archex/index`
- [Pipeline](https://github.com/mathews-tom/archex/tree/main/src/archex/pipeline): query assembly · Expands candidates, applies graph context, and enforces token budgets. · `src/archex/pipeline`
- [Serve](https://github.com/mathews-tom/archex/tree/main/src/archex/serve): context packing · Packages intent-aware context and renders JSON, Markdown, or XML. · `src/archex/serve`
- [Receipt](https://github.com/mathews-tom/archex/blob/main/src/archex/receipt.py): trust report · Explains freshness, completeness, skipped candidates, and omitted edges. · `src/archex/receipt.py`
- [Graph query](https://github.com/mathews-tom/archex/blob/main/src/archex/graph_query.py): context expansion · Graph-based lookup and multi-hop expansion used by retrieval and impact flows. · `src/archex/graph_query.py`
- [Languages](https://github.com/mathews-tom/archex/blob/main/src/archex/languages.py): adapter registry · Language tiering and adapter registration for parser extensibility. · `src/archex/languages.py`

### Delivery surfaces

User-facing entry points and integrations.

- [CLI](https://github.com/mathews-tom/archex/tree/main/src/archex/cli): command surface · Primary command-line front door for indexing, querying, analysis, and status. · `src/archex/cli`
- [Python API](https://github.com/mathews-tom/archex/blob/main/src/archex/api.py): library surface · Programmatic entry surface for embedding archex in other tools. · `src/archex/api.py`
- [MCP server](https://github.com/mathews-tom/archex/blob/main/src/archex/integrations/mcp.py): integration surface · Model Context Protocol serving surface for external agents. · `src/archex/integrations/mcp.py`
- [Docker](https://github.com/mathews-tom/archex/tree/main/docker): packaging surface · Container images for local deployment and portability. · `docker`

### Validation and observability

Benchmarks, metrics, and trust signals.

- [Metrics](https://github.com/mathews-tom/archex/tree/main/src/archex/metrics): local telemetry · Opt-in anonymous machine-local usage counters and health reporting. · `src/archex/metrics`
- [Benchmark](https://github.com/mathews-tom/archex/tree/main/src/archex/benchmark): evaluation harness · Task suites, baselines, gates, and regression reporting for quality control. · `src/archex/benchmark`
- [Doctor/status](https://github.com/mathews-tom/archex/blob/main/src/archex/doctor.py): health checks · Index health, grammar availability, freshness, and readiness reporting. · `src/archex/doctor.py`

### Local state

Repository-local artifacts under .archex/.

- [.archex](https://github.com/mathews-tom/archex/blob/main/.archex): repo-local state · Local settings, metadata, SQLite index data, vector artifacts, and graph artifacts. · `.archex`

## Connections

- Repo source → Acquire: discover · Repository inputs are discovered and normalized before processing.
- Acquire → Parse: parse · Acquired repositories are structurally extracted by language adapters.
- Parse → Analyze: derive signals · Parsed symbols, imports, and chunks feed higher-level analysis.
- Analyze → Index: index · Analysis enriches ranking, graph relationships, and stored artifacts.
- Index → Graph query: expand · Graph relationships are used to broaden and refine candidates.
- Index → Pipeline: ranked candidates · Search and rerank produce scored candidates for assembly.
- Graph query → Pipeline: context enrich · Graph/type context is merged into the query pipeline.
- Pipeline → Serve: bundle · Token-budgeted context bundles are prepared for presentation.
- Serve → Receipt: emit receipt · The final envelope includes provenance and omission details.
- Pipeline → Receipt: track omissions · Skipped items and budget decisions are recorded in the receipt.
- Serve → CLI: render · CLI exposes the packed context and report outputs.
- Serve → Python API: serve · Python callers consume the same context assembly pipeline.
- Serve → MCP server: serve · MCP exposes the same retrieval and rendering capabilities.
- Serve → Docker: package · Docker images deliver the runtime as a portable local tool.
- Index → .archex: persist · Indexes, metadata, and artifacts are stored in repo-local state.
- Metrics → Doctor/status: report · Metrics and health outputs contribute to operational visibility.
- Benchmark → Index: evaluate · Benchmarks measure retrieval and ranking quality of the core index.
- Benchmark → Pipeline: evaluate · Benchmark tasks also exercise bundle assembly and query flow.
- Benchmark → Doctor/status: gate · Readiness and regression gates summarize trust signals.
- Languages → Parse: register adapters · Language adapters are extended through registry-style hooks.
- Doctor/status → CLI: surface · Health and status checks are primarily exposed through the CLI.
