# vinsongqj/42_rag_against_the_machine architecture

> Architecture diagram of the GitHub repository vinsongqj/42_rag_against_the_machine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vinsongqj/42_rag_against_the_machine
- Repository: https://github.com/vinsongqj/42_rag_against_the_machine

## Overview

RAG against the machine ingests a corpus, chunks and indexes it, retrieves ranked source locations, and can generate grounded answers or evaluate retrieval recall. Users initiate work through the CLI or optional HTTP API; search supports BM25, semantic, and hybrid retrieval, with query caching. The graph preserves separate ingestion, chunking, indexing, ranking, generation, and evaluation stages. Edges reflect README workflows and shown calls/import references; several CLI and batch details are documented but their implementations were not fully sampled.

Main entry points: [`src/retrieve.py`](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/retrieve.py) (Source retrieval), [`src/__main__.py`](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/__main__.py) (CLI commands), [`src/api.py`](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/api.py) (HTTP API), [`src/index.py`](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/index.py) (Index builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User entry points"]
  node_cli["CLI commands<br/>[__main__.py]"]
  node_http_api["HTTP API<br/>[api.py]"]
end

subgraph group_indexing["Corpus indexing"]
  node_ingest["File ingestion<br/>[ingest.py]"]
  node_chunk["Chunking<br/>[chunk.py]"]
  node_index_builder["Index builder<br/>[index.py]"]
  node_bm25_store[("BM25 index<br/>[index.py]")]
  node_semantic[("Semantic index<br/>[semantic.py]")]
end

subgraph group_search["Retrieval and ranking"]
  node_retrieval["Source retrieval<br/>[retrieve.py]"]
  node_ranking["Search ranking<br/>[rank.py]"]
  node_query_cache[("Query cache<br/>[cache.py]")]
  node_index_version["Index versioning<br/>[version.py]"]
end

subgraph group_answer_eval["Answer and evaluation"]
  node_answering["Answer generation<br/>[generate.py]"]
  node_evaluation["Recall evaluation<br/>[evaluate.py]"]
  node_data_models["Data models<br/>[models.py]"]
end

node_cli_user(("CLI user"))
node_api_user(("HTTP client"))
node_corpus["Corpus files"]
node_ollama(("Ollama model"))

node_cli_user -->|"runs commands"| node_cli
node_api_user -->|"sends requests"| node_http_api
node_corpus -->|"provides files"| node_ingest
node_cli -->|"builds indexes"| node_index_builder
node_index_builder -->|"ingests corpus"| node_ingest
node_ingest -->|"chunks files"| node_chunk
node_index_builder -->|"writes index"| node_bm25_store
node_index_builder -->|"optionally builds"| node_semantic
node_semantic -->|"persists vectors"| node_semantic
node_cli -->|"runs search"| node_retrieval
node_retrieval -->|"loads index"| node_bm25_store
node_retrieval -->|"dispatches search"| node_ranking
node_ranking -->|"searches vectors"| node_semantic
node_retrieval -->|"reads and writes"| node_query_cache
node_retrieval -->|"checks versions"| node_index_version
node_cli -->|"generates answers"| node_answering
node_answering -->|"requests generation"| node_ollama
node_cli -->|"evaluates results"| node_evaluation
node_http_api -->|"retrieves sources"| node_retrieval
node_http_api -->|"generates answers"| node_answering
node_cli -->|"starts server"| node_http_api
node_ingest -->|"creates chunks"| node_data_models
node_retrieval -->|"returns sources"| node_data_models
node_http_api -->|"uses schemas"| node_data_models

click node_cli "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/__main__.py"
click node_http_api "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/api.py"
click node_ingest "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/ingest.py"
click node_chunk "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/chunk.py"
click node_index_builder "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/index.py"
click node_bm25_store "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/index.py"
click node_semantic "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/semantic.py"
click node_retrieval "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/retrieve.py"
click node_ranking "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/rank.py"
click node_query_cache "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/cache.py"
click node_index_version "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/version.py"
click node_answering "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/generate.py"
click node_evaluation "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/evaluate.py"
click node_data_models "https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/models.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_http_api,node_cli_user,node_api_user toneBlue
class node_ingest,node_chunk,node_index_builder,node_bm25_store,node_semantic toneAmber
class node_retrieval,node_ranking,node_query_cache,node_index_version toneMint
class node_answering,node_evaluation,node_data_models toneRose
class node_corpus,node_ollama toneIndigo
```

## Components

### User entry points

- [CLI commands](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/__main__.py): `src/__main__.py`
- [HTTP API](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/api.py): `src/api.py`

### Corpus indexing

- [File ingestion](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/ingest.py): `src/ingest.py`
- [Chunking](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/chunk.py): `src/chunk.py`
- [Index builder](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/index.py): `src/index.py`
- [BM25 index](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/index.py): `src/index.py`
- [Semantic index](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/semantic.py): `src/semantic.py`

### Retrieval and ranking

- [Source retrieval](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/retrieve.py): `src/retrieve.py`
- [Search ranking](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/rank.py): `src/rank.py`
- [Query cache](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/cache.py): `src/cache.py`
- [Index versioning](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/version.py): `src/version.py`

### Answer and evaluation

- [Answer generation](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/generate.py): `src/generate.py`
- [Recall evaluation](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/evaluate.py): `src/evaluate.py`
- [Data models](https://github.com/vinsongqj/42_rag_against_the_machine/blob/main/src/models.py): `src/models.py`

### Other components

- **CLI user**
- **HTTP client**
- **Corpus files**
- **Ollama model**

## Connections

- CLI user → CLI commands: runs commands (evidence: `README.md`)
- HTTP client → HTTP API: sends requests (evidence: `README.md`)
- Corpus files → File ingestion: provides files (evidence: `src/ingest.py`)
- CLI commands → Index builder: builds indexes (evidence: `src/__main__.py`)
- Index builder → File ingestion: ingests corpus (evidence: `src/index.py`)
- File ingestion → Chunking: chunks files (evidence: `src/ingest.py`)
- Index builder → BM25 index: writes index (evidence: `src/index.py`)
- Index builder → Semantic index: optionally builds (evidence: `src/index.py`)
- Semantic index → Semantic index: persists vectors (evidence: `src/semantic.py`)
- CLI commands → Source retrieval: runs search (evidence: `README.md`)
- Source retrieval → BM25 index: loads index (evidence: `src/retrieve.py`)
- Source retrieval → Search ranking: dispatches search (evidence: `src/retrieve.py`)
- Search ranking → Semantic index: searches vectors (evidence: `src/rank.py`)
- Source retrieval → Query cache: reads and writes (evidence: `src/retrieve.py`)
- Source retrieval → Index versioning: checks versions (evidence: `src/retrieve.py`)
- CLI commands → Answer generation: generates answers (evidence: `README.md`)
- Answer generation → Ollama model: requests generation (evidence: `src/generate.py`)
- CLI commands → Recall evaluation: evaluates results (evidence: `README.md`)
- HTTP API → Source retrieval: retrieves sources (evidence: `src/api.py`)
- HTTP API → Answer generation: generates answers (evidence: `src/api.py`)
- CLI commands → HTTP API: starts server (evidence: `src/__main__.py`)
- File ingestion → Data models: creates chunks (evidence: `src/ingest.py`)
- Source retrieval → Data models: returns sources (evidence: `src/retrieve.py`)
- HTTP API → Data models: uses schemas (evidence: `src/api.py`)