How rares8921/enterprise-multimodal-rag-platform works
This is a multiservice document-intelligence and RAG prototype. Its main workflow ingests tenant documents, stores metadata and raw files, processes OCR and layout asynchronously, indexes chunks, then answers authenticated queries through retrieval and LLM orchestration. Monitoring and LayoutLM fine-tuning are distinct supporting capabilities. The README documents service boundaries and workflow; source excerpts confirm several internal handoffs. Other links are shown only where the README or SOURCE INDEX supports them, so unsampled details are not inferred.
Main entry points: ingestion/main.py Upload API · inference-api/main.py Inference API · worker.py OCR worker · llm-orchestrator/main.py LLM orchestrator
Components 32
Document ingestion
- Upload API
services/ingestion/main.py - Document metadata
services/ingestion/models.py - Raw object storage
- Task queues
services/ingestion/task_queue.py
Processing and indexing
- OCR worker
services/ingestion/worker.py - EasyOCR engine
services/ingestion/ocr_engine.py - Layout parser worker
services/layout-parser/main.py - LayoutLM parser
services/layout-parser/LayoutParser.py - Embedding worker
services/embedding/main.py - Chunking and embeddings
services/embedding/EmbeddingGenerator.py - Tenant vector index
services/embedding/VectorStore.py
Query and answers
- Inference API
services/inference-api/main.py - Authentication and limits
services/inference-api/auth/dependencies.py - Hybrid retrieval
services/inference-api/utils/hybrid_retrieval.py - Context construction
services/inference-api/utils/context_utility.py - LLM orchestrator
services/llm-orchestrator/main.py - Prompt selection
services/llm-orchestrator/prompt_manager.py - Complexity scoring
services/llm-orchestrator/complexity_analyzer.py - Model routing
services/llm-orchestrator/utils/ModelRouter.py - Gemini and Mistral
services/llm-orchestrator/model_wrapper/GeminiLLM.py
Model workflow
- LayoutLM fine-tuning
ml_pipeline/kubeflow/layoutlm_pipeline.py - Training logic
ml_pipeline/kubeflow/layoutlm_logic.py
Monitoring
- Monitoring control plane
services/monitoring/control_plane/app.py - Monitoring event store
services/monitoring/event_store/db.py - Drift detection
services/monitoring/drift_detection/DriftDetector.py - RAG monitoring signals
services/monitoring/modules/rag_monitoring/signals.py
Other components
- Document uploader
- Query user
- Redis
- PostgreSQL
- MinIO
- Pinecone
Connections 33
- Document uploader → Upload API: uploads document
- Upload API → Document metadata: writes metadata
services/ingestion/main.py - Upload API → Raw object storage: stores file
- Upload API → Task queues: queues OCR
services/ingestion/main.py - Upload API → Redis: stores status
services/ingestion/main.py - Upload API → PostgreSQL: persists metadata
services/ingestion/main.py - OCR worker → Task queues: dequeues work
services/ingestion/worker.py - OCR worker → Raw object storage: reads file
services/ingestion/worker.py - OCR worker → EasyOCR engine: runs OCR
services/ingestion/worker.py - OCR worker → Redis: writes OCR data
- OCR worker → Task queues: queues layout
- Layout parser worker → Redis: reads OCR data
services/layout-parser/main.py - Layout parser worker → LayoutLM parser: parses layout
services/layout-parser/main.py - Layout parser worker → Redis: stores layout
services/layout-parser/main.py - Layout parser worker → Task queues: queues embedding
services/layout-parser/main.py - Embedding worker → Redis: reads OCR and layout
services/embedding/main.py - Embedding worker → Chunking and embeddings: chunks and embeds
services/embedding/main.py - Embedding worker → Tenant vector index: upserts vectors
services/embedding/main.py - Tenant vector index → Pinecone: stores vectors
- Query user → Inference API: submits query
- Inference API → Authentication and limits: checks access
services/inference-api/main.py - Inference API → Hybrid retrieval: reranks candidates
services/inference-api/main.py - Inference API → Pinecone: retrieves vectors
- Inference API → Context construction: builds context
- Inference API → LLM orchestrator: requests generation
- LLM orchestrator → Prompt selection: selects prompt
services/llm-orchestrator/main.py - LLM orchestrator → Complexity scoring: scores query
services/llm-orchestrator/main.py - LLM orchestrator → Model routing: routes model
services/llm-orchestrator/main.py - LLM orchestrator → Gemini and Mistral: calls provider
services/llm-orchestrator/main.py - LayoutLM fine-tuning → Training logic: uses training logic
ml_pipeline/kubeflow/layoutlm_pipeline.py - Monitoring control plane → Monitoring event store: writes events
services/monitoring/control_plane/app.py - Monitoring control plane → Drift detection: detects drift
services/monitoring/control_plane/app.py - Monitoring control plane → RAG monitoring signals: reads signals
services/monitoring/control_plane/app.py