# lu-yanru/42-rag_against_the_machine architecture

> Architecture diagram of the GitHub repository lu-yanru/42-rag_against_the_machine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lu-yanru/42-rag_against_the_machine
- Repository: https://github.com/lu-yanru/42-rag_against_the_machine

## Overview

This repository is documented as a RAG system for answering questions about a vLLM codebase. Its workflow indexes source files, retrieves relevant spans by lexical, semantic, or hybrid search, then either saves search results or generates and evaluates answers. The README describes the major components and their relationships, but their `src/` implementation files and processed/output data paths are absent from the supplied tree; those nodes therefore have no asserted source path.

Main entry points: [`data/raw/vllm-0.10.1`](https://github.com/lu-yanru/42-rag_against_the_machine/tree/main/data/raw/vllm-0.10.1) (vLLM source corpus)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_indexing["Indexing"]
  node_loader["File loader"]
  node_chunkers["Text and AST chunkers"]
  node_indexer["Incremental indexer"]
  node_lexindex[("BM25 index")]
  node_semindex[("Embedding index")]
  node_embedmodel["Sentence transformer"]
end

subgraph group_retrieval["Retrieval"]
  node_cache[("Query cache")]
  node_lexretriever["BM25 retriever"]
  node_semretriever["Semantic retriever"]
  node_hybrid["Hybrid RRF"]
  node_sources["Source locations"]
  node_searchresults[("Search results")]
end

subgraph group_answers["Answers and evaluation"]
  node_generator["Answer generator"]
  node_llm["Qwen3 model"]
  node_answerresults[("Answers and sources")]
  node_evaluator["Recall evaluator"]
end

node_user(("Questioner"))
node_corpus["vLLM source corpus<br/>[vllm-0.10.1]"]
node_cli["CLI"]
node_answeredquestions["Reference answers"]

node_cli -->|"indexes files"| node_loader
node_loader -->|"reads files"| node_corpus
node_loader -->|"provides text"| node_chunkers
node_chunkers -->|"provides chunks"| node_indexer
node_indexer -->|"builds BM25"| node_lexindex
node_indexer -->|"builds embeddings"| node_semindex
node_indexer -.->|"encodes chunks"| node_embedmodel
node_user -->|"submits question"| node_cli
node_cli -->|"checks cache"| node_cache
node_cli -->|"dispatches lexical"| node_lexretriever
node_cli -->|"dispatches semantic"| node_semretriever
node_cli -->|"dispatches hybrid"| node_hybrid
node_lexretriever -->|"searches index"| node_lexindex
node_semretriever -->|"searches vectors"| node_semindex
node_semretriever -.->|"encodes query"| node_embedmodel
node_hybrid -->|"combines rankings"| node_lexretriever
node_hybrid -->|"combines rankings"| node_semretriever
node_lexretriever -->|"returns locations"| node_sources
node_semretriever -->|"returns locations"| node_sources
node_hybrid -->|"returns locations"| node_sources
node_cli -->|"stores results"| node_cache
node_cache -->|"returns cached locations"| node_sources
node_cli -->|"saves search"| node_searchresults
node_cli -->|"requests answer"| node_generator
node_generator -->|"reads spans"| node_sources
node_generator -->|"reopens source"| node_corpus
node_generator -.->|"generates response"| node_llm
node_generator -->|"saves answer"| node_answerresults
node_cli -->|"runs evaluation"| node_evaluator
node_evaluator -->|"scores retrieval"| node_searchresults
node_evaluator -->|"compares references"| node_answeredquestions

click node_corpus "https://github.com/lu-yanru/42-rag_against_the_machine/tree/main/data/raw/vllm-0.10.1"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loader,node_chunkers,node_indexer,node_lexindex,node_semindex,node_embedmodel toneBlue
class node_cache,node_lexretriever,node_semretriever,node_hybrid,node_sources,node_searchresults toneAmber
class node_generator,node_llm,node_answerresults,node_evaluator toneMint
class node_user,node_cli,node_answeredquestions toneIndigo
class node_corpus toneTeal
```

## Components

### Indexing

- **File loader**
- **Text and AST chunkers**
- **Incremental indexer**
- **BM25 index**
- **Embedding index**
- **Sentence transformer**

### Retrieval

- **Query cache**
- **BM25 retriever**
- **Semantic retriever**
- **Hybrid RRF**
- **Source locations**
- **Search results**

### Answers and evaluation

- **Answer generator**
- **Qwen3 model**
- **Answers and sources**
- **Recall evaluator**

### Other components

- **Questioner**
- [vLLM source corpus](https://github.com/lu-yanru/42-rag_against_the_machine/tree/main/data/raw/vllm-0.10.1): `data/raw/vllm-0.10.1`
- **CLI**
- **Reference answers**

## Connections

- CLI → File loader: indexes files
- File loader → vLLM source corpus: reads files
- File loader → Text and AST chunkers: provides text
- Text and AST chunkers → Incremental indexer: provides chunks
- Incremental indexer → BM25 index: builds BM25
- Incremental indexer → Embedding index: builds embeddings
- Incremental indexer → Sentence transformer: encodes chunks
- Questioner → CLI: submits question
- CLI → Query cache: checks cache
- CLI → BM25 retriever: dispatches lexical
- CLI → Semantic retriever: dispatches semantic
- CLI → Hybrid RRF: dispatches hybrid
- BM25 retriever → BM25 index: searches index
- Semantic retriever → Embedding index: searches vectors
- Semantic retriever → Sentence transformer: encodes query
- Hybrid RRF → BM25 retriever: combines rankings
- Hybrid RRF → Semantic retriever: combines rankings
- BM25 retriever → Source locations: returns locations
- Semantic retriever → Source locations: returns locations
- Hybrid RRF → Source locations: returns locations
- CLI → Query cache: stores results
- Query cache → Source locations: returns cached locations
- CLI → Search results: saves search
- CLI → Answer generator: requests answer
- Answer generator → Source locations: reads spans
- Answer generator → vLLM source corpus: reopens source
- Answer generator → Qwen3 model: generates response
- Answer generator → Answers and sources: saves answer
- CLI → Recall evaluator: runs evaluation
- Recall evaluator → Search results: scores retrieval
- Recall evaluator → Reference answers: compares references