# yahyaoui7/true_rag architecture

> Architecture diagram of the GitHub repository yahyaoui7/true_rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yahyaoui7/true_rag
- Repository: https://github.com/yahyaoui7/true_rag

## Overview

RAG Against the Machine indexes a source corpus, retrieves ranked evidence, and generates grounded answers through CLI or local HTTP entry points. The map preserves separate lexical and semantic retrieval, hybrid fusion, incremental index maintenance, caching, dataset workflows, and evaluation. Edges follow README architecture or shown calls; several capabilities are documented but their implementations were not sampled, so the graph avoids inferring additional internal links.

Main entry points: [`src/__main__.py`](https://github.com/yahyaoui7/true_rag/blob/master/src/__main__.py) (CLI Commands), [`src/retriever.py`](https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py) (Search Coordinator), [`src/indexer.py`](https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py) (Index Manager), [`src/server.py`](https://github.com/yahyaoui7/true_rag/blob/master/src/server.py) (Local HTTP API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Corpus Indexing"]
  node_scanner["File Scanner<br/>[scanner.py]"]
  node_router["Chunk Dispatch<br/>[indexer.py]"]
  node_python["Python Chunker<br/>[python_chunker.py]"]
  node_markdown["Markdown Chunker"]
  node_text["Text Chunker<br/>[text_chunker.py]"]
  node_source["Chunk Metadata<br/>[models.py]"]
  node_indexer["Index Manager<br/>[indexer.py]"]
  node_manifest[("Incremental Manifest<br/>[indexer.py]")]
  node_lexical[("BM25 Index<br/>[indexer.py]")]
  node_vectors[("ChromaDB Vectors<br/>[indexer.py]")]
end

subgraph group_search["Retrieval"]
  node_searcher["Search Coordinator<br/>[retriever.py]"]
  node_bm25search["Lexical Search<br/>[retriever.py]"]
  node_semantic["Semantic Search<br/>[retriever.py]"]
  node_fusion["Hybrid Fusion<br/>[retriever.py]"]
end

subgraph group_answer["Answering"]
  node_cache[("Answer Cache<br/>[caching.py]")]
  node_prompt["Prompt Builder<br/>[prompts.py]"]
  node_generator["Qwen Generator<br/>[generate.py]"]
  node_model["Qwen Model"]
end

subgraph group_interfaces["User Interfaces"]
  node_cli["CLI Commands<br/>[__main__.py]"]
  node_http["Local HTTP API<br/>[server.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_evaluator["Recall Evaluator<br/>[evaluator.py]"]
  node_datasets["Search Datasets<br/>[models.py]"]
end

node_user(("User"))
node_corpus["Source Corpus"]

node_user -->|"runs commands"| node_cli
node_user -->|"queries API"| node_http
node_corpus -->|"scans files"| node_scanner
node_cli -->|"builds indexes"| node_indexer
node_indexer -->|"discovers files"| node_scanner
node_indexer -->|"dispatches files"| node_router
node_router -->|"chunks Python"| node_python
node_router -->|"chunks Markdown"| node_markdown
node_router -->|"chunks text"| node_text
node_python -->|"creates chunks"| node_source
node_markdown -->|"creates chunks"| node_source
node_text -->|"creates chunks"| node_source
node_indexer -->|"tracks changes"| node_manifest
node_indexer -->|"builds and saves"| node_lexical
node_indexer -->|"embeds and upserts"| node_vectors
node_cli -->|"searches queries"| node_searcher
node_http -->|"searches queries"| node_searcher
node_searcher -->|"loads index"| node_lexical
node_searcher -->|"runs lexical search"| node_bm25search
node_searcher -->|"runs semantic search"| node_semantic
node_bm25search -->|"scores candidates"| node_fusion
node_semantic -->|"scores candidates"| node_fusion
node_searcher -->|"fuses results"| node_fusion
node_searcher -->|"queries vectors"| node_vectors
node_cli -->|"checks and saves answers"| node_cache
node_http -->|"checks and saves answers"| node_cache
node_cli -->|"builds context prompt"| node_prompt
node_http -->|"builds context prompt"| node_prompt
node_cli -->|"generates answer"| node_generator
node_http -->|"generates answer"| node_generator
node_generator -->|"loads and invokes"| node_model
node_cli -->|"processes datasets"| node_datasets
node_cli -->|"evaluates recall"| node_evaluator
node_evaluator -->|"scores results"| node_datasets

click node_cli "https://github.com/yahyaoui7/true_rag/blob/master/src/__main__.py"
click node_http "https://github.com/yahyaoui7/true_rag/blob/master/src/server.py"
click node_scanner "https://github.com/yahyaoui7/true_rag/blob/master/src/scanner.py"
click node_router "https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py"
click node_python "https://github.com/yahyaoui7/true_rag/blob/master/src/chunking/python_chunker.py"
click node_markdown "https://github.com/yahyaoui7/true_rag/blob/master/src/chunking/markdown_chunker.py"
click node_text "https://github.com/yahyaoui7/true_rag/blob/master/src/chunking/text_chunker.py"
click node_source "https://github.com/yahyaoui7/true_rag/blob/master/src/models.py"
click node_indexer "https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py"
click node_manifest "https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py"
click node_lexical "https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py"
click node_vectors "https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py"
click node_searcher "https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py"
click node_bm25search "https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py"
click node_semantic "https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py"
click node_fusion "https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py"
click node_cache "https://github.com/yahyaoui7/true_rag/blob/master/src/caching.py"
click node_prompt "https://github.com/yahyaoui7/true_rag/blob/master/src/prompts.py"
click node_generator "https://github.com/yahyaoui7/true_rag/blob/master/src/generate.py"
click node_evaluator "https://github.com/yahyaoui7/true_rag/blob/master/src/evaluator.py"
click node_datasets "https://github.com/yahyaoui7/true_rag/blob/master/src/models.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scanner,node_router,node_python,node_markdown,node_text,node_source,node_indexer,node_manifest,node_lexical,node_vectors,node_user toneBlue
class node_searcher,node_bm25search,node_semantic,node_fusion toneAmber
class node_cache,node_prompt,node_generator,node_model toneMint
class node_cli,node_http toneRose
class node_evaluator,node_datasets,node_corpus toneIndigo
```

## Components

### Corpus Indexing

- [File Scanner](https://github.com/yahyaoui7/true_rag/blob/master/src/scanner.py): `src/scanner.py`
- [Chunk Dispatch](https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py): `src/indexer.py`
- [Python Chunker](https://github.com/yahyaoui7/true_rag/blob/master/src/chunking/python_chunker.py): `src/chunking/python_chunker.py`
- [Markdown Chunker](https://github.com/yahyaoui7/true_rag/blob/master/src/chunking/markdown_chunker.py): `src/chunking/markdown_chunker.py`
- [Text Chunker](https://github.com/yahyaoui7/true_rag/blob/master/src/chunking/text_chunker.py): `src/chunking/text_chunker.py`
- [Chunk Metadata](https://github.com/yahyaoui7/true_rag/blob/master/src/models.py): `src/models.py`
- [Index Manager](https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py): `src/indexer.py`
- [Incremental Manifest](https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py): `src/indexer.py`
- [BM25 Index](https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py): `src/indexer.py`
- [ChromaDB Vectors](https://github.com/yahyaoui7/true_rag/blob/master/src/indexer.py): `src/indexer.py`

### Retrieval

- [Search Coordinator](https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py): `src/retriever.py`
- [Lexical Search](https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py): `src/retriever.py`
- [Semantic Search](https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py): `src/retriever.py`
- [Hybrid Fusion](https://github.com/yahyaoui7/true_rag/blob/master/src/retriever.py): `src/retriever.py`

### Answering

- [Answer Cache](https://github.com/yahyaoui7/true_rag/blob/master/src/caching.py): `src/caching.py`
- [Prompt Builder](https://github.com/yahyaoui7/true_rag/blob/master/src/prompts.py): `src/prompts.py`
- [Qwen Generator](https://github.com/yahyaoui7/true_rag/blob/master/src/generate.py): `src/generate.py`
- **Qwen Model**

### User Interfaces

- [CLI Commands](https://github.com/yahyaoui7/true_rag/blob/master/src/__main__.py): `src/__main__.py`
- [Local HTTP API](https://github.com/yahyaoui7/true_rag/blob/master/src/server.py): `src/server.py`

### Evaluation

- [Recall Evaluator](https://github.com/yahyaoui7/true_rag/blob/master/src/evaluator.py): `src/evaluator.py`
- [Search Datasets](https://github.com/yahyaoui7/true_rag/blob/master/src/models.py): `src/models.py`

### Other components

- **User**
- **Source Corpus**

## Connections

- User → CLI Commands: runs commands (evidence: `src/__main__.py`)
- User → Local HTTP API: queries API (evidence: `src/server.py`)
- Source Corpus → File Scanner: scans files
- CLI Commands → Index Manager: builds indexes (evidence: `src/__main__.py`)
- Index Manager → File Scanner: discovers files (evidence: `src/indexer.py`)
- Index Manager → Chunk Dispatch: dispatches files (evidence: `src/indexer.py`)
- Chunk Dispatch → Python Chunker: chunks Python (evidence: `src/indexer.py`)
- Chunk Dispatch → Markdown Chunker: chunks Markdown (evidence: `src/indexer.py`)
- Chunk Dispatch → Text Chunker: chunks text (evidence: `src/indexer.py`)
- Python Chunker → Chunk Metadata: creates chunks (evidence: `src/chunking/python_chunker.py`)
- Markdown Chunker → Chunk Metadata: creates chunks (evidence: `src/chunking/markdown_chunker.py`)
- Text Chunker → Chunk Metadata: creates chunks (evidence: `src/chunking/text_chunker.py`)
- Index Manager → Incremental Manifest: tracks changes (evidence: `src/indexer.py`)
- Index Manager → BM25 Index: builds and saves (evidence: `src/indexer.py`)
- Index Manager → ChromaDB Vectors: embeds and upserts (evidence: `src/indexer.py`)
- CLI Commands → Search Coordinator: searches queries (evidence: `src/__main__.py`)
- Local HTTP API → Search Coordinator: searches queries (evidence: `src/server.py`)
- Search Coordinator → BM25 Index: loads index (evidence: `src/retriever.py`)
- Search Coordinator → Lexical Search: runs lexical search (evidence: `src/retriever.py`)
- Search Coordinator → Semantic Search: runs semantic search (evidence: `src/retriever.py`)
- Lexical Search → Hybrid Fusion: scores candidates (evidence: `src/retriever.py`)
- Semantic Search → Hybrid Fusion: scores candidates (evidence: `src/retriever.py`)
- Search Coordinator → Hybrid Fusion: fuses results (evidence: `src/retriever.py`)
- Search Coordinator → ChromaDB Vectors: queries vectors (evidence: `src/retriever.py`)
- CLI Commands → Answer Cache: checks and saves answers (evidence: `src/__main__.py`)
- Local HTTP API → Answer Cache: checks and saves answers (evidence: `src/server.py`)
- CLI Commands → Prompt Builder: builds context prompt (evidence: `src/__main__.py`)
- Local HTTP API → Prompt Builder: builds context prompt (evidence: `src/server.py`)
- CLI Commands → Qwen Generator: generates answer (evidence: `src/__main__.py`)
- Local HTTP API → Qwen Generator: generates answer (evidence: `src/server.py`)
- Qwen Generator → Qwen Model: loads and invokes (evidence: `src/generate.py`)
- CLI Commands → Search Datasets: processes datasets (evidence: `src/__main__.py`)
- CLI Commands → Recall Evaluator: evaluates recall (evidence: `src/__main__.py`)
- Recall Evaluator → Search Datasets: scores results (evidence: `src/evaluator.py`)
