How tomdevtech/localragpipelinemodular works
This is a local restaurant-review RAG application: the CLI loads and indexes review CSVs, then accepts questions, retrieves relevant chunks, optionally reranks them, and supplies context to an Ollama LLM for an answer. Chunking is a configurable family of strategies; retrieval persists vectors in Chroma. The graph follows the shown CLI and pipeline calls. The README documents interactive querying and YAML configuration; unsampled strategy implementations are represented by their exact paths, without inferring their internal calls.
Main entry points: strategies.py Chunker factory · pipeline.py RAG pipeline · main.py CLI · retriever.py Vector retriever
Components 21
Ingestion and chunking
- Review CSV loader
src/loaders.py - Chunker factory
src/chunking/strategies.py - Chunker interface
src/chunking/base.py - Fixed-size chunker
src/chunking/fixed_size.py - Recursive chunker
src/chunking/recursive.py - Document chunker
src/chunking/document.py - Semantic chunker
src/chunking/semantic.py - Propositional chunker
src/chunking/propositional.py - Agentic chunker
src/chunking/agentic.py - Sentence utilities
src/chunking/utils.py
Query services
- Vector retriever
src/retrieval/retriever.py - Embedding reranker
src/reranking/reranker.py - Chroma store
Application runtime
- CLI
src/main.py - RAG pipeline
src/pipeline.py
Configuration
- Settings
src/config/settings.py
Other components
- User
- Review CSV
data/realistic_restaurent_reviews.csv - Ollama embeddings
- Ollama LLM
- Generated answer
Connections 25
- User → CLI: asks question
README.md - CLI → Settings: loads settings
src/main.py - CLI → Review CSV loader: loads CSV
src/main.py - Review CSV → Review CSV loader: provides rows
src/loaders.py - CLI → RAG pipeline: indexes and queries
src/main.py - RAG pipeline → Settings: reads options
src/pipeline.py - RAG pipeline → Chunker factory: selects chunker
src/pipeline.py - Chunker factory → Fixed-size chunker: dispatches to
src/chunking/strategies.py - Chunker factory → Recursive chunker: dispatches to
src/chunking/strategies.py - Chunker factory → Document chunker: dispatches to
src/chunking/strategies.py - Chunker factory → Semantic chunker: dispatches to
src/chunking/strategies.py - Chunker factory → Propositional chunker: dispatches to
src/chunking/strategies.py - Chunker factory → Agentic chunker: dispatches to
src/chunking/strategies.py - RAG pipeline → Vector retriever: indexes chunks
src/pipeline.py - Review CSV loader → RAG pipeline: provides documents
src/main.py - RAG pipeline → Vector retriever: searches query
src/pipeline.py - RAG pipeline → Embedding reranker: reranks candidates
src/pipeline.py - Vector retriever → Chroma store: reads and writes
src/retrieval/retriever.py - Vector retriever → Ollama embeddings: embeds documents
src/retrieval/retriever.py - Embedding reranker → Ollama embeddings: embeds candidates
src/reranking/reranker.py - Semantic chunker → Ollama embeddings: embeds sentences
src/chunking/semantic.py - Semantic chunker → Sentence utilities: splits sentences
src/chunking/semantic.py - Propositional chunker → Sentence utilities: splits sentences
src/chunking/propositional.py - CLI → Ollama LLM: generates answer
src/main.py - CLI → Generated answer: prints answer
src/main.py