# abhaygupta002/smart-rag-platform architecture

> Architecture diagram of the GitHub repository abhaygupta002/smart-rag-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhaygupta002/smart-rag-platform
- Repository: https://github.com/abhaygupta002/smart-rag-platform

## Overview

SmartRAG is a Spring Boot document-ingestion and question-answering application. Users submit PDFs or queries through its web studio/API. Ingestion is orchestrated through PDF parsing, semantic chunking, and vector indexing; queries retrieve relevant material and generate cited answers. The graph also shows job checkpoints, query safeguards, and resource tuning. README statements establish the documented storage and provider capabilities; source excerpts and the SOURCE INDEX support the shown code relationships. Some optional UI/API wiring is not visible in the excerpts and is marked unverified.

Main entry points: [`src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/RagGenerationService.java) (RAG Generation), [`src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java) (Ingestion API), [`src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java`](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java) (Pipeline Orchestrator), [`src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java`](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java) (PDF Parser)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion Pipeline"]
  node_orchestrator["Pipeline Orchestrator"]
  node_parser["PDF Parser"]
  node_pagebatch["Page Batching"]
  node_stripper["PDF Structure"]
  node_chunker["Semantic Chunker"]
  node_vector["Vector Indexing"]
end

subgraph group_query["Retrieval &amp; Answers"]
  node_generation["RAG Generation"]
  node_relevance["Relevance Filter"]
  node_cache["Query Cache"]
  node_guard["Concurrency Guard"]
  node_breaker["Circuit Breaker"]
  node_metrics["Query Metrics"]
end

subgraph group_state["State &amp; Storage"]
  node_chromadb[("ChromaDB")]
  node_fallback[("Disk Vector Store")]
  node_jobs[("Document Jobs<br/>[DocumentJob.java]")]
  node_repository["Job Repository"]
  node_checkpoint["Checkpoints"]
end

subgraph group_ops["Operations &amp; UI"]
  node_studio["Web Studio<br/>[app.js]"]
  node_api["Ingestion API"]
  node_hardware["Hardware Tuning"]
  node_resources["Resource Manager"]
  node_os["OS Detection"]
  node_performance["Performance Control"]
  node_settings["Pipeline Settings"]
end

node_user(("User"))
node_gemini["Gemini API"]
node_ollama["Ollama"]
node_extractive["Extractive Fallback"]

node_user -.->|"uses"| node_studio
node_studio -.->|"requests"| node_api
node_api -->|"starts pipeline"| node_orchestrator
node_orchestrator -->|"parses PDFs"| node_parser
node_parser -->|"batches pages"| node_pagebatch
node_pagebatch -->|"extracts structure"| node_stripper
node_orchestrator -->|"chunks documents"| node_chunker
node_orchestrator -->|"indexes chunks"| node_vector
node_api -->|"answers queries"| node_generation
node_generation -->|"searches vectors"| node_vector
node_generation -->|"filters results"| node_relevance
node_generation -->|"reads and writes"| node_cache
node_generation -->|"guards queries"| node_guard
node_generation -->|"checks failures"| node_breaker
node_guard -->|"records metrics"| node_metrics
node_cache -->|"records hits"| node_metrics
node_vector -->|"upserts and searches"| node_chromadb
node_vector -->|"falls back to"| node_fallback
node_generation -.->|"generates answers"| node_gemini
node_generation -.->|"generates answers"| node_ollama
node_generation -.->|"falls back to"| node_extractive
node_parser -->|"tracks progress"| node_checkpoint
node_checkpoint -->|"persists jobs"| node_repository
node_repository -->|"stores records"| node_jobs
node_orchestrator -->|"checks pressure"| node_resources
node_parser -->|"gates work"| node_resources
node_chunker -->|"manages memory"| node_resources
node_api -->|"tunes profile"| node_hardware
node_api -->|"sets OS override"| node_os
node_api -->|"controls performance"| node_performance
node_hardware -->|"updates limits"| node_settings
node_parser -->|"reads configuration"| node_settings
node_chunker -->|"reads configuration"| node_settings

click node_studio "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/resources/static/js/app.js"
click node_api "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java"
click node_orchestrator "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java"
click node_parser "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java"
click node_pagebatch "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/PageBatchProcessor.java"
click node_stripper "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/MarkdownStructuralPdfStripper.java"
click node_chunker "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java"
click node_vector "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/vectorstore/ChromaVectorStoreService.java"
click node_generation "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/RagGenerationService.java"
click node_relevance "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryRelevanceFilter.java"
click node_cache "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/RagCacheService.java"
click node_guard "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryConcurrencyGuard.java"
click node_breaker "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryCircuitBreaker.java"
click node_metrics "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryMetricsTracker.java"
click node_fallback "https://github.com/abhaygupta002/smart-rag-platform/tree/main/data/vector_storage"
click node_jobs "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/model/DocumentJob.java"
click node_repository "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/repository/DocumentRepository.java"
click node_checkpoint "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/CheckpointService.java"
click node_hardware "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/DynamicHardwareTuningService.java"
click node_resources "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/ResourceManager.java"
click node_os "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/OperatingSystemDetector.java"
click node_performance "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/PerformanceController.java"
click node_settings "https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/config/RagPipelineProperties.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_orchestrator,node_parser,node_pagebatch,node_stripper,node_chunker,node_vector,node_user toneBlue
class node_generation,node_relevance,node_cache,node_guard,node_breaker,node_metrics toneAmber
class node_chromadb,node_fallback,node_jobs,node_repository,node_checkpoint,node_gemini toneMint
class node_studio,node_api,node_hardware,node_resources,node_os,node_performance,node_settings toneRose
class node_ollama,node_extractive toneIndigo
```

## Components

### Ingestion Pipeline

- [Pipeline Orchestrator](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java): `src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java`
- [PDF Parser](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java): `src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java`
- [Page Batching](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/PageBatchProcessor.java): `src/main/java/com/nextgem/smartrag/parser/PageBatchProcessor.java`
- [PDF Structure](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/parser/MarkdownStructuralPdfStripper.java): `src/main/java/com/nextgem/smartrag/parser/MarkdownStructuralPdfStripper.java`
- [Semantic Chunker](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java): `src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java`
- [Vector Indexing](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/vectorstore/ChromaVectorStoreService.java): `src/main/java/com/nextgem/smartrag/vectorstore/ChromaVectorStoreService.java`

### Retrieval & Answers

- [RAG Generation](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/RagGenerationService.java): `src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`
- [Relevance Filter](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryRelevanceFilter.java): `src/main/java/com/nextgem/smartrag/query/QueryRelevanceFilter.java`
- [Query Cache](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/RagCacheService.java): `src/main/java/com/nextgem/smartrag/query/RagCacheService.java`
- [Concurrency Guard](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryConcurrencyGuard.java): `src/main/java/com/nextgem/smartrag/query/QueryConcurrencyGuard.java`
- [Circuit Breaker](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryCircuitBreaker.java): `src/main/java/com/nextgem/smartrag/query/QueryCircuitBreaker.java`
- [Query Metrics](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/query/QueryMetricsTracker.java): `src/main/java/com/nextgem/smartrag/query/QueryMetricsTracker.java`

### State & Storage

- **ChromaDB**
- [Disk Vector Store](https://github.com/abhaygupta002/smart-rag-platform/tree/main/data/vector_storage): `data/vector_storage`
- [Document Jobs](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/model/DocumentJob.java): `src/main/java/com/nextgem/smartrag/model/DocumentJob.java`
- [Job Repository](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/repository/DocumentRepository.java): `src/main/java/com/nextgem/smartrag/repository/DocumentRepository.java`
- [Checkpoints](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/CheckpointService.java): `src/main/java/com/nextgem/smartrag/service/CheckpointService.java`

### Operations & UI

- [Web Studio](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/resources/static/js/app.js): `src/main/resources/static/js/app.js`
- [Ingestion API](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java): `src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`
- [Hardware Tuning](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/DynamicHardwareTuningService.java): `src/main/java/com/nextgem/smartrag/service/DynamicHardwareTuningService.java`
- [Resource Manager](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/ResourceManager.java): `src/main/java/com/nextgem/smartrag/service/ResourceManager.java`
- [OS Detection](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/OperatingSystemDetector.java): `src/main/java/com/nextgem/smartrag/service/OperatingSystemDetector.java`
- [Performance Control](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/service/PerformanceController.java): `src/main/java/com/nextgem/smartrag/service/PerformanceController.java`
- [Pipeline Settings](https://github.com/abhaygupta002/smart-rag-platform/blob/main/src/main/java/com/nextgem/smartrag/config/RagPipelineProperties.java): `src/main/java/com/nextgem/smartrag/config/RagPipelineProperties.java`

### Other components

- **User**
- **Gemini API**
- **Ollama**
- **Extractive Fallback**

## Connections

- User → Web Studio: uses
- Web Studio → Ingestion API: requests
- Ingestion API → Pipeline Orchestrator: starts pipeline (evidence: `src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`)
- Pipeline Orchestrator → PDF Parser: parses PDFs (evidence: `src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java`)
- PDF Parser → Page Batching: batches pages (evidence: `src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java`)
- Page Batching → PDF Structure: extracts structure (evidence: `src/main/java/com/nextgem/smartrag/parser/PageBatchProcessor.java`)
- Pipeline Orchestrator → Semantic Chunker: chunks documents (evidence: `src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java`)
- Pipeline Orchestrator → Vector Indexing: indexes chunks (evidence: `src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java`)
- Ingestion API → RAG Generation: answers queries (evidence: `src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`)
- RAG Generation → Vector Indexing: searches vectors (evidence: `src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`)
- RAG Generation → Relevance Filter: filters results (evidence: `src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`)
- RAG Generation → Query Cache: reads and writes (evidence: `src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`)
- RAG Generation → Concurrency Guard: guards queries (evidence: `src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`)
- RAG Generation → Circuit Breaker: checks failures (evidence: `src/main/java/com/nextgem/smartrag/service/RagGenerationService.java`)
- Concurrency Guard → Query Metrics: records metrics (evidence: `src/main/java/com/nextgem/smartrag/query/QueryConcurrencyGuard.java`)
- Query Cache → Query Metrics: records hits (evidence: `src/main/java/com/nextgem/smartrag/query/RagCacheService.java`)
- Vector Indexing → ChromaDB: upserts and searches (evidence: `README.md`)
- Vector Indexing → Disk Vector Store: falls back to (evidence: `README.md`)
- RAG Generation → Gemini API: generates answers (evidence: `README.md`)
- RAG Generation → Ollama: generates answers (evidence: `README.md`)
- RAG Generation → Extractive Fallback: falls back to (evidence: `README.md`)
- PDF Parser → Checkpoints: tracks progress (evidence: `src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java`)
- Checkpoints → Job Repository: persists jobs (evidence: `src/main/java/com/nextgem/smartrag/service/CheckpointService.java`)
- Job Repository → Document Jobs: stores records (evidence: `src/main/java/com/nextgem/smartrag/repository/DocumentRepository.java`)
- Pipeline Orchestrator → Resource Manager: checks pressure (evidence: `src/main/java/com/nextgem/smartrag/orchestrator/RagPipelineOrchestrator.java`)
- PDF Parser → Resource Manager: gates work (evidence: `src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java`)
- Semantic Chunker → Resource Manager: manages memory (evidence: `src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java`)
- Ingestion API → Hardware Tuning: tunes profile (evidence: `src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`)
- Ingestion API → OS Detection: sets OS override (evidence: `src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`)
- Ingestion API → Performance Control: controls performance (evidence: `src/main/java/com/nextgem/smartrag/controller/RagIngestionController.java`)
- Hardware Tuning → Pipeline Settings: updates limits (evidence: `src/main/java/com/nextgem/smartrag/service/DynamicHardwareTuningService.java`)
- PDF Parser → Pipeline Settings: reads configuration (evidence: `src/main/java/com/nextgem/smartrag/parser/PdfParallelParserService.java`)
- Semantic Chunker → Pipeline Settings: reads configuration (evidence: `src/main/java/com/nextgem/smartrag/chunking/RagChunkingService.java`)
