How abhaygupta002/smart-rag-platform works
SmartRAG is a Spring Boot document-ingestion and question-answering application. Users submit PDFs or queries through its web studio/API. Ingestion is orchestrated through PDF parsing, semantic chunking, and vector indexing; queries retrieve relevant material and generate cited answers. The graph also shows job checkpoints, query safeguards, and resource tuning. README statements establish the documented storage and provider capabilities; source excerpts and the SOURCE INDEX support the shown code relationships. Some optional UI/API wiring is not visible in the excerpts and is marked unverified.
Main entry points: RagGenerationService.java RAG Generation · RagIngestionController.java Ingestion API · RagPipelineOrchestrator.java Pipeline Orchestrator · PdfParallelParserService.java PDF Parser
Components 28
Ingestion Pipeline
- Pipeline Orchestrator
src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java - PDF Parser
src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java - Page Batching
src/main/java/com/nextgem/smartrag/parser/PageBatchProcessor.java - PDF Structure
src/main/java/com/nextgem/smartrag/parser/MarkdownStructuralPdfStripper.java - Semantic Chunker
src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java - Vector Indexing
src/main/java/com/nextgem/smartrag/vectorstore/ChromaVectorStoreService.java
Retrieval & Answers
- RAG Generation
src/main/java/com/nextgem/smartrag/service/RagGenerationService.java - Relevance Filter
src/main/java/com/nextgem/smartrag/query/QueryRelevanceFilter.java - Query Cache
src/main/java/com/nextgem/smartrag/query/RagCacheService.java - Concurrency Guard
src/main/java/com/nextgem/smartrag/query/QueryConcurrencyGuard.java - Circuit Breaker
src/main/java/com/nextgem/smartrag/query/QueryCircuitBreaker.java - Query Metrics
src/main/java/com/nextgem/smartrag/query/QueryMetricsTracker.java
State & Storage
- ChromaDB
- Disk Vector Store
data/vector_storage - Document Jobs
src/main/java/com/nextgem/smartrag/model/DocumentJob.java - Job Repository
src/main/java/com/nextgem/smartrag/repository/DocumentRepository.java - Checkpoints
src/main/java/com/nextgem/smartrag/service/CheckpointService.java
Operations & UI
- Web Studio
src/main/resources/static/js/app.js - Ingestion API
src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java - Hardware Tuning
src/main/java/com/nextgem/smartrag/service/DynamicHardwareTuningService.java - Resource Manager
src/main/java/com/nextgem/smartrag/service/ResourceManager.java - OS Detection
src/main/java/com/nextgem/smartrag/service/OperatingSystemDetector.java - Performance Control
src/main/java/com/nextgem/smartrag/service/PerformanceController.java - Pipeline Settings
src/main/java/com/nextgem/smartrag/config/RagPipelineProperties.java
Other components
- User
- Gemini API
- Ollama
- Extractive Fallback
Connections 33
- User → Web Studio: uses
- Web Studio → Ingestion API: requests
- Ingestion API → Pipeline Orchestrator: starts pipeline
src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java - Pipeline Orchestrator → PDF Parser: parses PDFs
src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java - PDF Parser → Page Batching: batches pages
src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java - Page Batching → PDF Structure: extracts structure
src/main/java/com/nextgem/smartrag/parser/PageBatchProcessor.java - Pipeline Orchestrator → Semantic Chunker: chunks documents
src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java - Pipeline Orchestrator → Vector Indexing: indexes chunks
src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java - Ingestion API → RAG Generation: answers queries
src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java - RAG Generation → Vector Indexing: searches vectors
src/main/java/com/nextgem/smartrag/service/RagGenerationService.java - RAG Generation → Relevance Filter: filters results
src/main/java/com/nextgem/smartrag/service/RagGenerationService.java - RAG Generation → Query Cache: reads and writes
src/main/java/com/nextgem/smartrag/service/RagGenerationService.java - RAG Generation → Concurrency Guard: guards queries
src/main/java/com/nextgem/smartrag/service/RagGenerationService.java - RAG Generation → Circuit Breaker: checks failures
src/main/java/com/nextgem/smartrag/service/RagGenerationService.java - Concurrency Guard → Query Metrics: records metrics
src/main/java/com/nextgem/smartrag/query/QueryConcurrencyGuard.java - Query Cache → Query Metrics: records hits
src/main/java/com/nextgem/smartrag/query/RagCacheService.java - Vector Indexing → ChromaDB: upserts and searches
README.md - Vector Indexing → Disk Vector Store: falls back to
README.md - RAG Generation → Gemini API: generates answers
README.md - RAG Generation → Ollama: generates answers
README.md - RAG Generation → Extractive Fallback: falls back to
README.md - PDF Parser → Checkpoints: tracks progress
src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java - Checkpoints → Job Repository: persists jobs
src/main/java/com/nextgem/smartrag/service/CheckpointService.java - Job Repository → Document Jobs: stores records
src/main/java/com/nextgem/smartrag/repository/DocumentRepository.java - Pipeline Orchestrator → Resource Manager: checks pressure
src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java - PDF Parser → Resource Manager: gates work
src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java - Semantic Chunker → Resource Manager: manages memory
src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java - Ingestion API → Hardware Tuning: tunes profile
src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java - Ingestion API → OS Detection: sets OS override
src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java - Ingestion API → Performance Control: controls performance
src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java - Hardware Tuning → Pipeline Settings: updates limits
src/main/java/com/nextgem/smartrag/service/DynamicHardwareTuningService.java - PDF Parser → Pipeline Settings: reads configuration
src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java - Semantic Chunker → Pipeline Settings: reads configuration
src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java