# rajdeepofgithub/sec-filings-rag architecture

> Architecture diagram of the GitHub repository rajdeepofgithub/sec-filings-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rajdeepofgithub/sec-filings-rag
- Repository: https://github.com/rajdeepofgithub/sec-filings-rag

## Overview

SEC Filings RAG ingests filings and earnings-call transcripts, structures their content into section-aware chunks, indexes them for dense and sparse search, and answers questions with cited evidence. The principal query path combines retrieval, deduplication, rank fusion, reranking, grounded generation, citation checks, and a confidence signal. Separate evaluation and baseline artifacts measure behavior. The graph includes documented capabilities even where implementation excerpts are absent; those links cite the README, and unsampled implementation details are not inferred.

Main entry points: [`src/generator.py`](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/generator.py) (Grounded Answer Generator), [`src/loader.py`](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/loader.py) (Document Loader), [`src/pipeline.py`](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/pipeline.py) (Search Orchestrator), [`src/indexer.py`](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/indexer.py) (Index Builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Document Ingestion"]
  node_loader["Document Loader<br/>[loader.py]"]
  node_ir["Document IR<br/>[ir.py]"]
  node_table_profile["Table Profiler<br/>[table_profile.py]"]
  node_table_convert["Table Converter<br/>[table_convert.py]"]
  node_sections["Section Detection<br/>[sections.py]"]
  node_chunker["Structure-aware Chunker<br/>[chunker.py]"]
end

subgraph group_indexing["Corpus and Indexing"]
  node_corpus["Corpus Builder<br/>[corpus.py]"]
  node_embedder["Text Embedder<br/>[embedder.py]"]
  node_indexer["Index Builder<br/>[indexer.py]"]
end

subgraph group_search["Hybrid Search"]
  node_pipeline["Search Orchestrator<br/>[pipeline.py]"]
  node_retriever["Dense and Sparse Retrieval<br/>[retriever.py]"]
  node_fusion["Dedup and Rank Fusion<br/>[fusion.py]"]
  node_reranker["Cross-encoder Reranker<br/>[reranker.py]"]
end

subgraph group_answers["Answer and Verification"]
  node_generator["Grounded Answer Generator<br/>[generator.py]"]
  node_verification["Citation Verifier<br/>[verify.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_grading["Answer Grader<br/>[grading.py]"]
  node_baseline["Baseline Runs"]
end

node_questioner(("Questioner"))
node_filings["Filings and Transcripts"]
node_chroma[("ChromaDB")]
node_bm25[("BM25 Index")]
node_llm{{"OpenAI API"}}

node_filings -->|"loads"| node_loader
node_loader -->|"builds blocks"| node_ir
node_loader -->|"profiles tables"| node_table_profile
node_loader -->|"converts tables"| node_table_convert
node_corpus -->|"loads documents"| node_loader
node_corpus -->|"attributes sections"| node_sections
node_corpus -->|"chunks documents"| node_chunker
node_chunker -->|"reads blocks"| node_ir
node_indexer -->|"builds from"| node_corpus
node_indexer -->|"embeds chunks"| node_embedder
node_indexer -->|"writes vectors"| node_chroma
node_indexer -->|"builds index"| node_bm25
node_questioner -->|"submits query"| node_pipeline
node_pipeline -->|"runs searches"| node_retriever
node_retriever -->|"embeds query"| node_embedder
node_retriever -->|"queries vectors"| node_chroma
node_retriever -->|"queries terms"| node_bm25
node_pipeline -->|"deduplicates and fuses"| node_fusion
node_pipeline -->|"reranks candidates"| node_reranker
node_generator -->|"retrieves context"| node_pipeline
node_generator -->|"generates answer"| node_llm
node_generator -->|"checks citations"| node_verification
node_generator -->|"returns cited answer"| node_questioner
node_grading -->|"grades answers"| node_generator
node_baseline -.->|"provides eval cases"| node_grading

click node_loader "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/loader.py"
click node_ir "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/ir.py"
click node_table_profile "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/table_profile.py"
click node_table_convert "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/table_convert.py"
click node_sections "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/sections.py"
click node_chunker "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/chunker.py"
click node_corpus "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/corpus.py"
click node_embedder "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/embedder.py"
click node_indexer "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/indexer.py"
click node_pipeline "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/pipeline.py"
click node_retriever "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/retriever.py"
click node_fusion "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/fusion.py"
click node_reranker "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/reranker.py"
click node_generator "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/generator.py"
click node_verification "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/verify.py"
click node_grading "https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/grading.py"
click node_baseline "https://github.com/rajdeepofgithub/sec-filings-rag/tree/main/baseline"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loader,node_ir,node_table_profile,node_table_convert,node_sections,node_chunker toneBlue
class node_corpus,node_embedder,node_indexer,node_chroma,node_bm25 toneAmber
class node_pipeline,node_retriever,node_fusion,node_reranker,node_llm toneMint
class node_generator,node_verification toneRose
class node_grading,node_baseline,node_questioner,node_filings toneIndigo
```

## Components

### Document Ingestion

- [Document Loader](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/loader.py): `src/loader.py`
- [Document IR](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/ir.py): `src/ir.py`
- [Table Profiler](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/table_profile.py): `src/table_profile.py`
- [Table Converter](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/table_convert.py): `src/table_convert.py`
- [Section Detection](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/sections.py): `src/sections.py`
- [Structure-aware Chunker](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/chunker.py): `src/chunker.py`

### Corpus and Indexing

- [Corpus Builder](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/corpus.py): `src/corpus.py`
- [Text Embedder](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/embedder.py): `src/embedder.py`
- [Index Builder](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/indexer.py): `src/indexer.py`

### Hybrid Search

- [Search Orchestrator](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/pipeline.py): `src/pipeline.py`
- [Dense and Sparse Retrieval](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/retriever.py): `src/retriever.py`
- [Dedup and Rank Fusion](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/fusion.py): `src/fusion.py`
- [Cross-encoder Reranker](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/reranker.py): `src/reranker.py`

### Answer and Verification

- [Grounded Answer Generator](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/generator.py): `src/generator.py`
- [Citation Verifier](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/verify.py): `src/verify.py`

### Evaluation

- [Answer Grader](https://github.com/rajdeepofgithub/sec-filings-rag/blob/main/src/grading.py): `src/grading.py`
- [Baseline Runs](https://github.com/rajdeepofgithub/sec-filings-rag/tree/main/baseline): `baseline`

### Other components

- **Questioner**
- **Filings and Transcripts**
- **ChromaDB**
- **BM25 Index**
- **OpenAI API**

## Connections

- Filings and Transcripts → Document Loader: loads (evidence: `src/loader.py`)
- Document Loader → Document IR: builds blocks (evidence: `src/loader.py`)
- Document Loader → Table Profiler: profiles tables (evidence: `src/loader.py`)
- Document Loader → Table Converter: converts tables (evidence: `src/loader.py`)
- Corpus Builder → Document Loader: loads documents (evidence: `src/corpus.py`)
- Corpus Builder → Section Detection: attributes sections (evidence: `src/corpus.py`)
- Corpus Builder → Structure-aware Chunker: chunks documents (evidence: `src/corpus.py`)
- Structure-aware Chunker → Document IR: reads blocks (evidence: `src/chunker.py`)
- Index Builder → Corpus Builder: builds from (evidence: `src/indexer.py`)
- Index Builder → Text Embedder: embeds chunks (evidence: `src/indexer.py`)
- Index Builder → ChromaDB: writes vectors (evidence: `src/indexer.py`)
- Index Builder → BM25 Index: builds index (evidence: `src/indexer.py`)
- Questioner → Search Orchestrator: submits query (evidence: `src/pipeline.py`)
- Search Orchestrator → Dense and Sparse Retrieval: runs searches (evidence: `src/pipeline.py`)
- Dense and Sparse Retrieval → Text Embedder: embeds query (evidence: `src/retriever.py`)
- Dense and Sparse Retrieval → ChromaDB: queries vectors (evidence: `src/retriever.py`)
- Dense and Sparse Retrieval → BM25 Index: queries terms (evidence: `src/retriever.py`)
- Search Orchestrator → Dedup and Rank Fusion: deduplicates and fuses (evidence: `src/pipeline.py`)
- Search Orchestrator → Cross-encoder Reranker: reranks candidates (evidence: `src/pipeline.py`)
- Grounded Answer Generator → Search Orchestrator: retrieves context (evidence: `src/generator.py`)
- Grounded Answer Generator → OpenAI API: generates answer (evidence: `README.md`)
- Grounded Answer Generator → Citation Verifier: checks citations (evidence: `README.md`)
- Grounded Answer Generator → Questioner: returns cited answer (evidence: `README.md`)
- Answer Grader → Grounded Answer Generator: grades answers (evidence: `src/grading.py`)
- Baseline Runs → Answer Grader: provides eval cases (evidence: `README.md`)