4d4mkhalil/rag-against-the-machine

How 4d4mkhalil/rag-against-the-machine works

This is a local RAG application for answering questions about the vLLM codebase. A user invokes the CLI, whose Pipeline builds or reuses indexes, searches with hybrid BM25 and FAISS retrieval, and can augment retrieved context into an answer; batch search, answer generation, and recall@k evaluation are also supported. Chunking, index construction, retrieval, augmentation, and generation are distinct stages. The graph follows documented workflows and sampled calls; batch/evaluation details not visible in the excerpts are linked only where the README documents them.

Main entry points: pipeline.py RAG workflows · indexer.py Index construction · retriever.py Hybrid retrieval · settings.py Runtime settings

Components 18

CLI and orchestration

Command entry src/__main__.py
RAG workflows src/pipeline.py
Runtime settings src/settings.py
Data models src/models.py

Corpus indexing

Document chunking src/chunker.py
Index construction src/indexer.py
BM25 index
FAISS index
Chunk records

Hybrid retrieval

Hybrid retrieval src/retriever.py

Answering and evaluation

Context augmentation src/aug_generator.py
Local language model src/llm.py
Recall evaluation src/pipeline.py

Other components

CLI user
Source documents
Questioner
Question datasets
Search and answer files

Connections 28

CLI user → Command entry: invokes src/__main__.py
Command entry → RAG workflows: dispatches src/__main__.py
RAG workflows → Runtime settings: checks settings src/pipeline.py
RAG workflows → Index construction: builds index src/pipeline.py
Index construction → Document chunking: loads chunks src/indexer.py
Document chunking → Source documents: reads files src/chunker.py
Index construction → BM25 index: writes index src/indexer.py
Index construction → FAISS index: writes vectors src/indexer.py
Index construction → Chunk records: writes records src/indexer.py
Index construction → Data models: creates chunks src/indexer.py
RAG workflows → Hybrid retrieval: requests search src/pipeline.py
Hybrid retrieval → BM25 index: searches index src/retriever.py
Hybrid retrieval → FAISS index: searches vectors src/retriever.py
Hybrid retrieval → Chunk records: loads records src/retriever.py
Hybrid retrieval → Data models: returns chunks src/retriever.py
Questioner → RAG workflows: asks question
RAG workflows → Context augmentation: requests answer src/pipeline.py
Context augmentation → Local language model: generates text src/aug_generator.py
RAG workflows → Source documents: reads context src/pipeline.py
RAG workflows → Question datasets: loads questions README.md
RAG workflows → Search and answer files: writes outputs README.md
RAG workflows → Recall evaluation: evaluates recall README.md
Recall evaluation → Question datasets: compares ground truth README.md
Recall evaluation → Search and answer files: reads search results README.md
Index construction → Runtime settings: uses configuration src/indexer.py
Hybrid retrieval → Runtime settings: uses configuration src/retriever.py
Context augmentation → Runtime settings: uses defaults src/aug_generator.py
Local language model → Runtime settings: uses model config src/llm.py

Drawn by GitDiagram on November 14, 2026 from the repository's file tree, README and source files. Read as Markdown ·