# tarunima-ds/rbi_ai_policy_assistant architecture

> Architecture diagram of the GitHub repository tarunima-ds/rbi_ai_policy_assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tarunima-ds/rbi_ai_policy_assistant
- Repository: https://github.com/tarunima-ds/rbi_ai_policy_assistant

## Overview

This repository is an RBI-document question-answering assistant with a separate PDF indexing workflow. An operator can index a PDF from the documents folder; ingestion extracts pages, prepares paragraphs and chunks, creates embeddings, and saves a FAISS index. A user asks a question in the Streamlit chat, which retrieves relevant chunks and sends them with the question to Gemini for an answer. The source index also identifies text-cleaning and metadata helpers, but their relationship to the main ingestion path is not shown. The README is empty, so unsampled capabilities and wiring are not inferred.

Main entry points: [`pipeline/ingestion.py`](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/pipeline/ingestion.py) (Ingestion pipeline), [`app.py`](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/app.py) (Indexing entry), [`services/text_extractor.py`](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/text_extractor.py) (Text extractor), [`pipeline/rag.py`](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/pipeline/rag.py) (RAG pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Document ingestion"]
  node_ingest_app["Indexing entry<br/>[app.py]"]
  node_ing_pipeline["Ingestion pipeline<br/>[ingestion.py]"]
  node_logger["Logger setup<br/>[logger.py]"]
end

subgraph group_prep["Text processing"]
  node_pdf_loader["PDF loader<br/>[pdf_loader.py]"]
  node_validator["File validator<br/>[file_validator.py]"]
  node_paragraph_processor["Paragraph processor"]
  node_paragraph_detector["Paragraph detector"]
  node_chunker["Chunker<br/>[chunker.py]"]
  node_embedding["Embedding generator<br/>[embedding.py]"]
  node_text_extractor["Text extractor<br/>[text_extractor.py]"]
  node_cleaner["Text cleaner<br/>[cleaner.py]"]
  node_header_footer["Header footer remover<br/>[header_footer.py]"]
  node_metadata["Metadata extractor<br/>[metadata.py]"]
end

subgraph group_search["Retrieval and generation"]
  node_vectorstore[("FAISS vector store<br/>[vectorstore.py]")]
  node_retriever["Retriever<br/>[retriever.py]"]
  node_rag["RAG pipeline<br/>[rag.py]"]
  node_llm["Gemini answer service<br/>[llm.py]"]
end

subgraph group_ui["User interface"]
  node_dashboard["Streamlit chat<br/>[app.py]"]
end

node_operator(("Operator"))
node_corpus["RBI PDF corpus"]
node_gemini["Gemini API"]
node_user(("Question asker"))

node_operator -->|"runs indexing"| node_ingest_app
node_ingest_app -->|"selects PDFs"| node_corpus
node_ingest_app -->|"starts pipeline"| node_ing_pipeline
node_ingest_app -->|"configures logging"| node_logger
node_ing_pipeline -->|"extracts pages"| node_pdf_loader
node_pdf_loader -->|"validates file"| node_validator
node_ing_pipeline -->|"processes pages"| node_paragraph_processor
node_paragraph_processor -->|"splits paragraphs"| node_paragraph_detector
node_ing_pipeline -->|"chunks paragraphs"| node_chunker
node_ing_pipeline -->|"embeds chunks"| node_embedding
node_ing_pipeline -->|"builds and saves"| node_vectorstore
node_text_extractor -->|"uses loader"| node_pdf_loader
node_text_extractor -->|"uses metadata"| node_metadata
node_text_extractor -->|"cleans text"| node_cleaner
node_cleaner -->|"uses remover"| node_header_footer
node_user -->|"asks question"| node_dashboard
node_dashboard -->|"calls ask"| node_rag
node_rag -->|"searches context"| node_retriever
node_retriever -->|"searches index"| node_vectorstore
node_rag -->|"requests answer"| node_llm
node_llm -->|"generates content"| node_gemini

click node_corpus "https://github.com/tarunima-ds/rbi_ai_policy_assistant/tree/main/documents"
click node_ingest_app "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/app.py"
click node_ing_pipeline "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/pipeline/ingestion.py"
click node_pdf_loader "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/pdf_loader.py"
click node_validator "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/file_validator.py"
click node_paragraph_processor "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/paragraph_processor.py"
click node_paragraph_detector "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/paragraph_detector.py"
click node_chunker "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/chunker.py"
click node_embedding "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/embedding.py"
click node_text_extractor "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/text_extractor.py"
click node_cleaner "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/cleaner.py"
click node_header_footer "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/header_footer.py"
click node_metadata "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/metadata.py"
click node_vectorstore "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/vectorstore.py"
click node_retriever "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/retriever.py"
click node_rag "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/pipeline/rag.py"
click node_llm "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/llm.py"
click node_dashboard "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/dashboard/app.py"
click node_logger "https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/utils/logger.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_app,node_ing_pipeline,node_logger toneBlue
class node_pdf_loader,node_validator,node_paragraph_processor,node_paragraph_detector,node_chunker,node_embedding,node_text_extractor,node_cleaner,node_header_footer,node_metadata toneAmber
class node_vectorstore,node_retriever,node_rag,node_llm,node_gemini toneMint
class node_dashboard toneRose
class node_operator,node_user toneIndigo
class node_corpus toneTeal
```

## Components

### Document ingestion

- [Indexing entry](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/app.py): `app.py`
- [Ingestion pipeline](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/pipeline/ingestion.py): `pipeline/ingestion.py`
- [Logger setup](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/utils/logger.py): `utils/logger.py`

### Text processing

- [PDF loader](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/pdf_loader.py): `services/pdf_loader.py`
- [File validator](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/file_validator.py): `services/file_validator.py`
- [Paragraph processor](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/paragraph_processor.py): `services/paragraph_processor.py`
- [Paragraph detector](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/paragraph_detector.py): `services/paragraph_detector.py`
- [Chunker](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/chunker.py): `services/chunker.py`
- [Embedding generator](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/embedding.py): `services/embedding.py`
- [Text extractor](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/text_extractor.py): `services/text_extractor.py`
- [Text cleaner](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/cleaner.py): `services/cleaner.py`
- [Header footer remover](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/header_footer.py): `services/header_footer.py`
- [Metadata extractor](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/metadata.py): `services/metadata.py`

### Retrieval and generation

- [FAISS vector store](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/vectorstore.py): `services/vectorstore.py`
- [Retriever](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/retriever.py): `services/retriever.py`
- [RAG pipeline](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/pipeline/rag.py): `pipeline/rag.py`
- [Gemini answer service](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/services/llm.py): `services/llm.py`

### User interface

- [Streamlit chat](https://github.com/tarunima-ds/rbi_ai_policy_assistant/blob/main/dashboard/app.py): `dashboard/app.py`

### Other components

- **Operator**
- [RBI PDF corpus](https://github.com/tarunima-ds/rbi_ai_policy_assistant/tree/main/documents): `documents`
- **Gemini API**
- **Question asker**

## Connections

- Operator → Indexing entry: runs indexing
- Indexing entry → RBI PDF corpus: selects PDFs (evidence: `app.py`)
- Indexing entry → Ingestion pipeline: starts pipeline (evidence: `app.py`)
- Indexing entry → Logger setup: configures logging (evidence: `app.py`)
- Ingestion pipeline → PDF loader: extracts pages (evidence: `pipeline/ingestion.py`)
- PDF loader → File validator: validates file (evidence: `services/pdf_loader.py`)
- Ingestion pipeline → Paragraph processor: processes pages (evidence: `pipeline/ingestion.py`)
- Paragraph processor → Paragraph detector: splits paragraphs (evidence: `services/paragraph_processor.py`)
- Ingestion pipeline → Chunker: chunks paragraphs (evidence: `pipeline/ingestion.py`)
- Ingestion pipeline → Embedding generator: embeds chunks (evidence: `pipeline/ingestion.py`)
- Ingestion pipeline → FAISS vector store: builds and saves (evidence: `pipeline/ingestion.py`)
- Text extractor → PDF loader: uses loader (evidence: `services/text_extractor.py`)
- Text extractor → Metadata extractor: uses metadata (evidence: `services/text_extractor.py`)
- Text extractor → Text cleaner: cleans text (evidence: `services/text_extractor.py`)
- Text cleaner → Header footer remover: uses remover (evidence: `services/cleaner.py`)
- Question asker → Streamlit chat: asks question (evidence: `dashboard/app.py`)
- Streamlit chat → RAG pipeline: calls ask (evidence: `dashboard/app.py`)
- RAG pipeline → Retriever: searches context (evidence: `pipeline/rag.py`)
- Retriever → FAISS vector store: searches index (evidence: `services/retriever.py`)
- RAG pipeline → Gemini answer service: requests answer (evidence: `pipeline/rag.py`)
- Gemini answer service → Gemini API: generates content (evidence: `services/llm.py`)
