# shubham-pande-ai/financial-rag architecture

> Architecture diagram of the GitHub repository shubham-pande-ai/financial-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shubham-pande-ai/financial-rag
- Repository: https://github.com/shubham-pande-ai/financial-rag

## Overview

Financial RAG ingests Indian-company annual reports and earnings-call transcripts, then answers research questions with cited text and financial figures. Its main workflow runs from document collection and extraction through chunking, embedding, and indexing; queries enter through the API, retrieve and rerank evidence, then pass through intent decomposition, SQL/vector fetching, fusion, prompt construction, and LLM generation. The graph also preserves evaluation as a documented product capability. Some ingestion and API connections are based on README descriptions because their implementation was not excerpted.

Main entry points: [`synthesis/pipeline.py`](https://github.com/shubham-pande-ai/financial-rag/blob/main/synthesis/pipeline.py) (Synthesis orchestration), [`pipeline/retrieval/retriever.py`](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/retrieval/retriever.py) (Hybrid retrieval), [`rag/rag_engine.py`](https://github.com/shubham-pande-ai/financial-rag/blob/main/rag/rag_engine.py) (RAG answer engine), [`schema_bridge/schema_bridge.py`](https://github.com/shubham-pande-ai/financial-rag/blob/main/schema_bridge/schema_bridge.py) (SQL/vector bridge)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Document ingestion"]
  node_scraper["Company scraper"]
  node_extractor["PDF extraction<br/>[pdf_extractor.py]"]
  node_cleaner["Text cleaning<br/>[text_cleaner.py]"]
  node_chunker["Document chunking<br/>[chunker.py]"]
  node_embedder["FinLang embeddings<br/>[embedder.py]"]
  node_qdrant_loader["Vector indexing<br/>[qdrant_loader.py]"]
end

subgraph group_stores["Data stores"]
  node_object_store[("PDF object storage")]
  node_qdrant[("Qdrant vectors")]
  node_mysql[("MySQL financial data<br/>[database.py]")]
end

subgraph group_query["Research query"]
  node_ui["Streamlit interface<br/>[app.py]"]
  node_api["FastAPI service<br/>[server.py]"]
  node_rag["RAG answer engine<br/>[rag_engine.py]"]
  node_retriever["Hybrid retrieval<br/>[retriever.py]"]
  node_reranker["Cross-encoder reranking<br/>[reranker.py]"]
end

subgraph group_answer["Evidence and answers"]
  node_synthesis["Synthesis orchestration<br/>[pipeline.py]"]
  node_decomposer["Question decomposition"]
  node_bridge["SQL/vector bridge<br/>[schema_bridge.py]"]
  node_fusion["Evidence reconciliation<br/>[fusion_layer.py]"]
  node_prompt["Citation prompt building<br/>[prompt_builder.py]"]
  node_evaluation["Retrieval evaluation<br/>[eval_suite.py]"]
end

node_researcher(("Researcher"))
node_llm["LLM provider"]

node_scraper -->|"stores PDFs"| node_object_store
node_scraper -->|"logs metadata"| node_mysql
node_object_store -->|"supplies PDFs"| node_extractor
node_extractor -->|"extracts text"| node_cleaner
node_cleaner -->|"cleans content"| node_chunker
node_chunker -->|"prepares chunks"| node_embedder
node_embedder -->|"embeds chunks"| node_qdrant_loader
node_qdrant_loader -->|"indexes vectors"| node_qdrant
node_researcher -->|"asks question"| node_ui
node_ui -->|"sends query"| node_api
node_api -->|"routes query"| node_rag
node_rag -->|"retrieves evidence"| node_retriever
node_retriever -->|"embeds query"| node_embedder
node_retriever -->|"searches vectors"| node_qdrant_loader
node_retriever -->|"ranks candidates"| node_reranker
node_reranker -->|"passes top chunks"| node_synthesis
node_rag -->|"runs synthesis"| node_synthesis
node_synthesis -->|"decomposes intent"| node_decomposer
node_synthesis -->|"fetches evidence"| node_bridge
node_bridge -->|"queries financials"| node_mysql
node_bridge -->|"fetches text"| node_retriever
node_synthesis -->|"reconciles results"| node_fusion
node_synthesis -->|"builds prompt"| node_prompt
node_rag -->|"generates answer"| node_llm
node_evaluation -->|"evaluates retrieval"| node_retriever
node_evaluation -->|"evaluates ranking"| node_reranker

click node_scraper "https://github.com/shubham-pande-ai/financial-rag/blob/main/screener_downloader.py"
click node_extractor "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/extract/pdf_extractor.py"
click node_cleaner "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/extract/text_cleaner.py"
click node_chunker "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/loader/chunker.py"
click node_embedder "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/loader/embedder.py"
click node_qdrant_loader "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/loader/qdrant_loader.py"
click node_mysql "https://github.com/shubham-pande-ai/financial-rag/blob/main/db/database.py"
click node_ui "https://github.com/shubham-pande-ai/financial-rag/blob/main/app.py"
click node_api "https://github.com/shubham-pande-ai/financial-rag/blob/main/server.py"
click node_rag "https://github.com/shubham-pande-ai/financial-rag/blob/main/rag/rag_engine.py"
click node_retriever "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/retrieval/retriever.py"
click node_reranker "https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/retrieval/reranker.py"
click node_synthesis "https://github.com/shubham-pande-ai/financial-rag/blob/main/synthesis/pipeline.py"
click node_decomposer "https://github.com/shubham-pande-ai/financial-rag/blob/main/decomposer/atomic_decomposer.py"
click node_bridge "https://github.com/shubham-pande-ai/financial-rag/blob/main/schema_bridge/schema_bridge.py"
click node_fusion "https://github.com/shubham-pande-ai/financial-rag/blob/main/fusion/fusion_layer.py"
click node_prompt "https://github.com/shubham-pande-ai/financial-rag/blob/main/synthesis/prompt_builder.py"
click node_evaluation "https://github.com/shubham-pande-ai/financial-rag/blob/main/eval/eval_suite.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scraper,node_extractor,node_cleaner,node_chunker,node_embedder,node_qdrant_loader toneBlue
class node_object_store,node_qdrant,node_mysql toneAmber
class node_ui,node_api,node_rag,node_retriever,node_reranker toneMint
class node_synthesis,node_decomposer,node_bridge,node_fusion,node_prompt,node_evaluation toneRose
class node_researcher,node_llm toneIndigo
```

## Components

### Document ingestion

- [Company scraper](https://github.com/shubham-pande-ai/financial-rag/blob/main/screener_downloader.py): `screener_downloader.py`
- [PDF extraction](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/extract/pdf_extractor.py): `pipeline/extract/pdf_extractor.py`
- [Text cleaning](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/extract/text_cleaner.py): `pipeline/extract/text_cleaner.py`
- [Document chunking](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/loader/chunker.py): `pipeline/loader/chunker.py`
- [FinLang embeddings](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/loader/embedder.py): `pipeline/loader/embedder.py`
- [Vector indexing](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/loader/qdrant_loader.py): `pipeline/loader/qdrant_loader.py`

### Data stores

- **PDF object storage**
- **Qdrant vectors**
- [MySQL financial data](https://github.com/shubham-pande-ai/financial-rag/blob/main/db/database.py): `db/database.py`

### Research query

- [Streamlit interface](https://github.com/shubham-pande-ai/financial-rag/blob/main/app.py): `app.py`
- [FastAPI service](https://github.com/shubham-pande-ai/financial-rag/blob/main/server.py): `server.py`
- [RAG answer engine](https://github.com/shubham-pande-ai/financial-rag/blob/main/rag/rag_engine.py): `rag/rag_engine.py`
- [Hybrid retrieval](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/retrieval/retriever.py): `pipeline/retrieval/retriever.py`
- [Cross-encoder reranking](https://github.com/shubham-pande-ai/financial-rag/blob/main/pipeline/retrieval/reranker.py): `pipeline/retrieval/reranker.py`

### Evidence and answers

- [Synthesis orchestration](https://github.com/shubham-pande-ai/financial-rag/blob/main/synthesis/pipeline.py): `synthesis/pipeline.py`
- [Question decomposition](https://github.com/shubham-pande-ai/financial-rag/blob/main/decomposer/atomic_decomposer.py): `decomposer/atomic_decomposer.py`
- [SQL/vector bridge](https://github.com/shubham-pande-ai/financial-rag/blob/main/schema_bridge/schema_bridge.py): `schema_bridge/schema_bridge.py`
- [Evidence reconciliation](https://github.com/shubham-pande-ai/financial-rag/blob/main/fusion/fusion_layer.py): `fusion/fusion_layer.py`
- [Citation prompt building](https://github.com/shubham-pande-ai/financial-rag/blob/main/synthesis/prompt_builder.py): `synthesis/prompt_builder.py`
- [Retrieval evaluation](https://github.com/shubham-pande-ai/financial-rag/blob/main/eval/eval_suite.py): `eval/eval_suite.py`

### Other components

- **Researcher**
- **LLM provider**

## Connections

- Company scraper → PDF object storage: stores PDFs (evidence: `Readme.md`)
- Company scraper → MySQL financial data: logs metadata (evidence: `Readme.md`)
- PDF object storage → PDF extraction: supplies PDFs (evidence: `Readme.md`)
- PDF extraction → Text cleaning: extracts text (evidence: `Readme.md`)
- Text cleaning → Document chunking: cleans content (evidence: `pipeline/loader/chunker.py`)
- Document chunking → FinLang embeddings: prepares chunks (evidence: `pipeline/loader/embedder.py`)
- FinLang embeddings → Vector indexing: embeds chunks (evidence: `pipeline/loader/qdrant_loader.py`)
- Vector indexing → Qdrant vectors: indexes vectors (evidence: `Readme.md`)
- Researcher → Streamlit interface: asks question (evidence: `Readme.md`)
- Streamlit interface → FastAPI service: sends query (evidence: `Readme.md`)
- FastAPI service → RAG answer engine: routes query (evidence: `server.py`)
- RAG answer engine → Hybrid retrieval: retrieves evidence (evidence: `rag/rag_engine.py`)
- Hybrid retrieval → FinLang embeddings: embeds query (evidence: `pipeline/retrieval/retriever.py`)
- Hybrid retrieval → Vector indexing: searches vectors (evidence: `pipeline/retrieval/retriever.py`)
- Hybrid retrieval → Cross-encoder reranking: ranks candidates (evidence: `Readme.md`)
- Cross-encoder reranking → Synthesis orchestration: passes top chunks (evidence: `synthesis/pipeline.py`)
- RAG answer engine → Synthesis orchestration: runs synthesis (evidence: `rag/rag_engine.py`)
- Synthesis orchestration → Question decomposition: decomposes intent (evidence: `synthesis/pipeline.py`)
- Synthesis orchestration → SQL/vector bridge: fetches evidence (evidence: `synthesis/pipeline.py`)
- SQL/vector bridge → MySQL financial data: queries financials (evidence: `schema_bridge/schema_bridge.py`)
- SQL/vector bridge → Hybrid retrieval: fetches text (evidence: `schema_bridge/schema_bridge.py`)
- Synthesis orchestration → Evidence reconciliation: reconciles results (evidence: `synthesis/pipeline.py`)
- Synthesis orchestration → Citation prompt building: builds prompt (evidence: `synthesis/pipeline.py`)
- RAG answer engine → LLM provider: generates answer (evidence: `Readme.md`)
- Retrieval evaluation → Hybrid retrieval: evaluates retrieval (evidence: `eval/eval_suite.py`)
- Retrieval evaluation → Cross-encoder reranking: evaluates ranking (evidence: `eval/eval_suite.py`)
