How elproximoframework/2026-06-16-rag-industrial works
This is an industrial RAG application for querying technical and maintenance documents, ingesting files, and synchronizing document directories. The main user flow is an API query routed through semantic caching and an agentic RAG pipeline to retrieval, context compression, and grounded generation. Separate API capabilities support direct ingestion, incremental sync, health checks, and evaluation. The source index and excerpts support the main orchestration links; unsampled implementation details are represented only where module references establish the subsystem.
Main entry points: pipeline.py Ingestion pipeline · routes.py API routes · agentic_pipeline.py Agentic RAG pipeline · engine.py Hybrid retrieval engine
Components 24
API and operations
- FastAPI app
app/api/main.py - API routes
app/api/routes.py - Pipeline setup
app/api/dependencies.py - API schemas
app/api/schemas.py - RAGAS evaluation
app/api/routes.py
Document ingestion
- Ingestion pipeline
app/ingestion/pipeline.py - PDF and DOCX parsers
app/ingestion/parsers - Parent-child chunking
app/ingestion/chunking/splitter.py - Metadata extraction
app/ingestion/metadata/extractor.py
Retrieval and answers
- Agentic RAG pipeline
app/retrieval/agentic_pipeline.py - Query router
app/retrieval/router.py - Hybrid retrieval engine
app/retrieval/engine.py - BM25 search
app/retrieval/bm25.py - Query transformation
app/retrieval/transformation.py - Cross-encoder reranker
app/retrieval/reranker.py - Context compressor
app/retrieval/compressor.py - Grounded generator
app/retrieval/generator.py - Semantic cache
app/retrieval/cache.py
Stores and sync
- Directory sync
app/ingestion/sync/orchestrator.py - Sync ledger
app/ingestion/sync/ledger.py - Parent document store
app/ingestion/storage/doc_store.py - Vector store
app/ingestion/storage/vector_store.py
Other components
- API client
- Gemini API
Connections 36
- API client → FastAPI app: sends requests
app/api/main.py - FastAPI app → API routes: mounts routes
app/api/main.py - FastAPI app → Pipeline setup: initializes pipelines
app/api/main.py - API routes → API schemas: uses schemas
app/api/routes.py - API routes → Pipeline setup: gets dependencies
app/api/routes.py - Pipeline setup → Ingestion pipeline: initializes
app/api/dependencies.py - Pipeline setup → Agentic RAG pipeline: initializes
app/api/dependencies.py - Pipeline setup → Semantic cache: initializes
app/api/dependencies.py - API routes → Agentic RAG pipeline: queries
app/api/routes.py - API routes → Semantic cache: checks and updates
app/api/routes.py - API routes → Ingestion pipeline: ingests files
app/api/routes.py - API routes → Directory sync: synchronizes directories
app/api/routes.py - API routes → RAGAS evaluation: runs evaluation
app/api/routes.py - RAGAS evaluation → Agentic RAG pipeline: queries dataset
app/api/routes.py - RAGAS evaluation → Gemini API: evaluates with
app/api/routes.py - Ingestion pipeline → PDF and DOCX parsers: parses documents
app/ingestion/pipeline.py - Ingestion pipeline → Parent-child chunking: splits text
app/ingestion/pipeline.py - Ingestion pipeline → Metadata extraction: extracts metadata
app/ingestion/pipeline.py - Ingestion pipeline → Parent document store: stores parents
app/ingestion/pipeline.py - Ingestion pipeline → Vector store: indexes children
app/ingestion/pipeline.py - Ingestion pipeline → Sync ledger: updates ledger
app/ingestion/pipeline.py - Directory sync → Ingestion pipeline: processes changed files
app/ingestion/sync/orchestrator.py - Directory sync → Sync ledger: detects and records changes
app/ingestion/sync/orchestrator.py - Directory sync → Parent document store: deletes parents
app/ingestion/sync/orchestrator.py - Directory sync → Vector store: deletes vectors
app/ingestion/sync/orchestrator.py - Agentic RAG pipeline → Query router: classifies query
app/retrieval/agentic_pipeline.py - Agentic RAG pipeline → Hybrid retrieval engine: retrieves candidates
app/retrieval/agentic_pipeline.py - Agentic RAG pipeline → Context compressor: compresses context
app/retrieval/agentic_pipeline.py - Agentic RAG pipeline → Grounded generator: generates answer
app/retrieval/agentic_pipeline.py - Hybrid retrieval engine → BM25 search: searches lexically
app/retrieval/engine.py - Hybrid retrieval engine → Query transformation: transforms queries
app/retrieval/engine.py - Hybrid retrieval engine → Cross-encoder reranker: reranks results
app/retrieval/engine.py - Hybrid retrieval engine → Vector store: searches vectors
app/retrieval/engine.py - Hybrid retrieval engine → Ingestion pipeline: retrieves parent chunks
app/retrieval/engine.py - Context compressor → Cross-encoder reranker: uses reranker
app/retrieval/compressor.py - Query router → Gemini API: classifies with
app/retrieval/router.py