How bilalerrabia/rag works
RAG against the Machine indexes a vLLM repository, retrieves relevant source spans for a question, and uses those spans to generate a grounded answer. Its CLI also supports batch search/answer workflows and Recall@k evaluation against supplied datasets. The graph separates ingestion, retrieval, generation, and evaluation, with typed data models as a shared internal layer. The README documents the dual-index design; edges are limited to relationships shown in sampled source or explicitly described there.
Main entry points: __main__.py RAG CLI · indexer.py Dual Indexer · retriever.py Hybrid Search · evaluator.py Recall Evaluator
Components 20
Ingestion
- File Chunker
src/chunker.py - Dual Indexer
src/indexer.py - Chunk Models
src/models.py - Chunk Metadata
- BM25 Index
- ChromaDB Collection
- Embedding Model
Retrieval
- Hybrid Search
src/retriever.py - Source Files
- Source Models
src/models.py - Context Loader
src/__main__.py
Generation
- Answer Builder
src/answerer.py - Qwen Generator
src/llm_model.py
Evaluation
- Search Results
src/models.py - Question Datasets
data/datasets/UnansweredQuestions/dataset_docs_public.json - Recall Evaluator
src/evaluator.py - Answered Datasets
data/datasets/AnsweredQuestions/dataset_docs_public.json
Other components
- User
- vLLM Repository
- RAG CLI
src/__main__.py
Connections 25
- User → RAG CLI: asks or runs
src/__main__.py - vLLM Repository → Dual Indexer: provides files
README.md - RAG CLI → Dual Indexer: invokes indexing
src/__main__.py - Dual Indexer → File Chunker: loads chunks
src/indexer.py - File Chunker → Chunk Models: creates records
src/chunker.py - Dual Indexer → Chunk Models: uses records
src/indexer.py - Dual Indexer → ChromaDB Collection: writes vectors
src/indexer.py - Dual Indexer → Embedding Model: encodes chunks
src/indexer.py - Dual Indexer → BM25 Index: writes index
src/indexer.py - Dual Indexer → Chunk Metadata: writes metadata
src/indexer.py - RAG CLI → Hybrid Search: searches query
src/__main__.py - Hybrid Search → BM25 Index: retrieves terms
src/retriever.py - Hybrid Search → Chunk Metadata: loads chunk metadata
src/retriever.py - Hybrid Search → ChromaDB Collection: queries vectors
src/retriever.py - Hybrid Search → Embedding Model: encodes query
src/retriever.py - Hybrid Search → Source Models: returns spans
src/retriever.py - RAG CLI → Context Loader: loads answer context
src/__main__.py - Context Loader → Source Files: reads spans
src/__main__.py - RAG CLI → Answer Builder: requests answer
src/__main__.py - Answer Builder → Qwen Generator: generates response
src/answerer.py - RAG CLI → Question Datasets: loads questions
src/__main__.py - RAG CLI → Search Results: records search output
src/__main__.py - RAG CLI → Recall Evaluator: runs metrics
src/__main__.py - Recall Evaluator → Search Results: reads retrievals
src/evaluator.py - Recall Evaluator → Answered Datasets: compares sources
src/evaluator.py