How shubham-pande-ai/financial-rag works
Financial RAG ingests Indian-company annual reports and earnings-call transcripts, then answers research questions with cited text and financial figures. Its main workflow runs from document collection and extraction through chunking, embedding, and indexing; queries enter through the API, retrieve and rerank evidence, then pass through intent decomposition, SQL/vector fetching, fusion, prompt construction, and LLM generation. The graph also preserves evaluation as a documented product capability. Some ingestion and API connections are based on README descriptions because their implementation was not excerpted.
Main entry points: pipeline.py Synthesis orchestration · retriever.py Hybrid retrieval · rag_engine.py RAG answer engine · schema_bridge.py SQL/vector bridge
Components 22
Document ingestion
- Company scraper
screener_downloader.py - PDF extraction
pipeline/extract/pdf_extractor.py - Text cleaning
pipeline/extract/text_cleaner.py - Document chunking
pipeline/loader/chunker.py - FinLang embeddings
pipeline/loader/embedder.py - Vector indexing
pipeline/loader/qdrant_loader.py
Data stores
- PDF object storage
- Qdrant vectors
- MySQL financial data
db/database.py
Research query
- Streamlit interface
app.py - FastAPI service
server.py - RAG answer engine
rag/rag_engine.py - Hybrid retrieval
pipeline/retrieval/retriever.py - Cross-encoder reranking
pipeline/retrieval/reranker.py
Evidence and answers
- Synthesis orchestration
synthesis/pipeline.py - Question decomposition
decomposer/atomic_decomposer.py - SQL/vector bridge
schema_bridge/schema_bridge.py - Evidence reconciliation
fusion/fusion_layer.py - Citation prompt building
synthesis/prompt_builder.py - Retrieval evaluation
eval/eval_suite.py
Other components
- Researcher
- LLM provider
Connections 26
- Company scraper → PDF object storage: stores PDFs
Readme.md - Company scraper → MySQL financial data: logs metadata
Readme.md - PDF object storage → PDF extraction: supplies PDFs
Readme.md - PDF extraction → Text cleaning: extracts text
Readme.md - Text cleaning → Document chunking: cleans content
pipeline/loader/chunker.py - Document chunking → FinLang embeddings: prepares chunks
pipeline/loader/embedder.py - FinLang embeddings → Vector indexing: embeds chunks
pipeline/loader/qdrant_loader.py - Vector indexing → Qdrant vectors: indexes vectors
Readme.md - Researcher → Streamlit interface: asks question
Readme.md - Streamlit interface → FastAPI service: sends query
Readme.md - FastAPI service → RAG answer engine: routes query
server.py - RAG answer engine → Hybrid retrieval: retrieves evidence
rag/rag_engine.py - Hybrid retrieval → FinLang embeddings: embeds query
pipeline/retrieval/retriever.py - Hybrid retrieval → Vector indexing: searches vectors
pipeline/retrieval/retriever.py - Hybrid retrieval → Cross-encoder reranking: ranks candidates
Readme.md - Cross-encoder reranking → Synthesis orchestration: passes top chunks
synthesis/pipeline.py - RAG answer engine → Synthesis orchestration: runs synthesis
rag/rag_engine.py - Synthesis orchestration → Question decomposition: decomposes intent
synthesis/pipeline.py - Synthesis orchestration → SQL/vector bridge: fetches evidence
synthesis/pipeline.py - SQL/vector bridge → MySQL financial data: queries financials
schema_bridge/schema_bridge.py - SQL/vector bridge → Hybrid retrieval: fetches text
schema_bridge/schema_bridge.py - Synthesis orchestration → Evidence reconciliation: reconciles results
synthesis/pipeline.py - Synthesis orchestration → Citation prompt building: builds prompt
synthesis/pipeline.py - RAG answer engine → LLM provider: generates answer
Readme.md - Retrieval evaluation → Hybrid retrieval: evaluates retrieval
eval/eval_suite.py - Retrieval evaluation → Cross-encoder reranking: evaluates ranking
eval/eval_suite.py