# aryaa-pawar/rtc-insight-engine architecture

> Architecture diagram of the GitHub repository aryaa-pawar/rtc-insight-engine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aryaa-pawar/rtc-insight-engine
- Repository: https://github.com/aryaa-pawar/rtc-insight-engine

## Overview

RTC Insight Engine combines a report-processing pipeline, an AI question-answering workflow, and analytics dashboards. Users ask questions through the FastAPI app; its handler calls the RAG engine, which searches report chunks, builds source context, and can use Gemini to generate an answer. Separately, report PDFs are extracted, chunked, and processed into analytics datasets. The README documents the pipeline and dashboard capabilities; several pipeline stages and the dashboard implementation are not shown in source excerpts, so those links rely on README evidence.

Main entry points: [`data/processed`](https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed) (Master dataset), [`chatbot/rag_engine.py`](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py) (RAG engine), [`app/main.py`](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/app/main.py) (Web app), [`app/templates/index.html`](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/app/templates/index.html) (Question page)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Question Interface"]
  node_webapp["Web app<br/>[main.py]"]
  node_page["Question page<br/>[index.html]"]
end

subgraph group_assistant["AI Assistant"]
  node_rag["RAG engine<br/>[rag_engine.py]"]
  node_retrieval["TF-IDF retrieval<br/>[rag_engine.py]"]
  node_answering["Answer generation<br/>[rag_engine.py]"]
end

subgraph group_pipeline["Data Pipeline"]
  node_reports["RTC report PDFs"]
  node_extraction["PDF extraction<br/>[pdf_extractor.py]"]
  node_masterdata[("Master dataset")]
  node_cleaning["Metric cleaning"]
  node_metrics["Metrics extraction"]
  node_chunks["Text chunking<br/>[generate_chunks.py]"]
  node_chunkdata[("Report chunks")]
  node_facts["RTC facts"]
end

subgraph group_analytics["Analytics"]
  node_dashboard["Looker Studio"]
  node_analyticsdata[("Analytics datasets")]
end

node_users(("RTC users"))
node_gemini{{"Gemini"}}
node_mysql[("MySQL")]

node_users -->|"ask questions"| node_page
node_page -.->|"submits question"| node_webapp
node_webapp -->|"calls"| node_rag
node_rag -->|"retrieves chunks"| node_retrieval
node_retrieval -->|"reads"| node_chunkdata
node_rag -->|"builds answer"| node_answering
node_answering -->|"generates with"| node_gemini
node_answering -->|"returns sourced answer"| node_users
node_reports -->|"extracts text"| node_extraction
node_extraction -->|"produces"| node_masterdata
node_masterdata -->|"cleans metrics"| node_cleaning
node_masterdata -->|"extracts metrics"| node_metrics
node_masterdata -->|"creates chunks"| node_chunks
node_chunks -->|"writes"| node_chunkdata
node_masterdata -.->|"derives facts"| node_facts
node_chunkdata -->|"indexes for search"| node_retrieval
node_cleaning -->|"prepares datasets"| node_analyticsdata
node_metrics -->|"produces metrics"| node_analyticsdata
node_analyticsdata -->|"powers dashboards"| node_dashboard
node_masterdata -->|"loads reports"| node_mysql
node_chunkdata -->|"loads chunks"| node_mysql
node_analyticsdata -->|"loads metrics"| node_mysql

click node_webapp "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/app/main.py"
click node_page "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/app/templates/index.html"
click node_rag "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py"
click node_retrieval "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py"
click node_answering "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py"
click node_reports "https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/raw_pdfs"
click node_extraction "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/pdf_extractor.py"
click node_masterdata "https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed"
click node_cleaning "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/clean_and_normalize.py"
click node_metrics "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/extract_named_metrics.py"
click node_chunks "https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/generate_chunks.py"
click node_chunkdata "https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed"
click node_facts "https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed"
click node_analyticsdata "https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webapp,node_page,node_users toneBlue
class node_rag,node_retrieval,node_answering,node_mysql toneAmber
class node_reports,node_extraction,node_masterdata,node_cleaning,node_metrics,node_chunks,node_chunkdata,node_facts toneMint
class node_dashboard,node_analyticsdata toneRose
class node_gemini toneIndigo
```

## Components

### Question Interface

- [Web app](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/app/main.py): `app/main.py`
- [Question page](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/app/templates/index.html): `app/templates/index.html`

### AI Assistant

- [RAG engine](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py): `chatbot/rag_engine.py`
- [TF-IDF retrieval](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py): `chatbot/rag_engine.py`
- [Answer generation](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/chatbot/rag_engine.py): `chatbot/rag_engine.py`

### Data Pipeline

- [RTC report PDFs](https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/raw_pdfs): `data/raw_pdfs`
- [PDF extraction](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/pdf_extractor.py): `scripts/pdf_extractor.py`
- [Master dataset](https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed): `data/processed`
- [Metric cleaning](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/clean_and_normalize.py): `scripts/clean_and_normalize.py`
- [Metrics extraction](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/extract_named_metrics.py): `scripts/extract_named_metrics.py`
- [Text chunking](https://github.com/aryaa-pawar/rtc-insight-engine/blob/main/scripts/generate_chunks.py): `scripts/generate_chunks.py`
- [Report chunks](https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed): `data/processed`
- [RTC facts](https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed): `data/processed`

### Analytics

- **Looker Studio**
- [Analytics datasets](https://github.com/aryaa-pawar/rtc-insight-engine/tree/main/data/processed): `data/processed`

### Other components

- **RTC users**
- **Gemini**
- **MySQL**

## Connections

- RTC users → Question page: ask questions (evidence: `README.md`)
- Question page → Web app: submits question
- Web app → RAG engine: calls (evidence: `app/main.py`)
- RAG engine → TF-IDF retrieval: retrieves chunks (evidence: `chatbot/rag_engine.py`)
- TF-IDF retrieval → Report chunks: reads (evidence: `chatbot/rag_engine.py`)
- RAG engine → Answer generation: builds answer (evidence: `chatbot/rag_engine.py`)
- Answer generation → Gemini: generates with (evidence: `chatbot/rag_engine.py`)
- Answer generation → RTC users: returns sourced answer (evidence: `README.md`)
- RTC report PDFs → PDF extraction: extracts text (evidence: `README.md`)
- PDF extraction → Master dataset: produces (evidence: `README.md`)
- Master dataset → Metric cleaning: cleans metrics (evidence: `README.md`)
- Master dataset → Metrics extraction: extracts metrics (evidence: `README.md`)
- Master dataset → Text chunking: creates chunks (evidence: `README.md`)
- Text chunking → Report chunks: writes (evidence: `README.md`)
- Master dataset → RTC facts: derives facts
- Report chunks → TF-IDF retrieval: indexes for search (evidence: `README.md`)
- Metric cleaning → Analytics datasets: prepares datasets (evidence: `README.md`)
- Metrics extraction → Analytics datasets: produces metrics (evidence: `README.md`)
- Analytics datasets → Looker Studio: powers dashboards (evidence: `README.md`)
- Master dataset → MySQL: loads reports
- Report chunks → MySQL: loads chunks
- Analytics datasets → MySQL: loads metrics