# pritam-parali/document-parsing-using-rag-major-project architecture

> Architecture diagram of the GitHub repository pritam-parali/document-parsing-using-rag-major-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pritam-parali/document-parsing-using-rag-major-project
- Repository: https://github.com/pritam-parali/document-parsing-using-rag-major-project

## Overview

This is a Streamlit document assistant: users upload supported files, ask questions, and receive CSV analysis or retrieval-grounded LLM answers. Its separate YouTube workflow fetches transcripts and produces downloadable notes; OCR and Mermaid diagrams provide additional document/screenshot capabilities. The main RAG, persistence, and YouTube modules are sampled; some UI wiring is omitted, so uncertain links are not shown.

Main entry points: [`app.py`](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py) (Streamlit app), [`src/search.py`](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/search.py) (RAG search), [`src/vectorestore.py`](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/vectorestore.py) (FAISS store), [`src/youtube_loader.py`](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_loader.py) (Transcript loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["User interface"]
  node_app["Streamlit app<br/>[app.py]"]
  node_history["Chat history<br/>[chat_history.py]"]
  node_historyfile[("Conversation JSON<br/>[chat_history.json]")]
end

subgraph group_rag["Document RAG"]
  node_loader["File loader<br/>[data_loader.py]"]
  node_embed["Chunk and embed<br/>[embedding.py]"]
  node_store[("FAISS store<br/>[vectorestore.py]")]
  node_search["RAG search<br/>[search.py]"]
  node_csv["CSV analysis<br/>[csv_agent.py]"]
end

subgraph group_youtube["YouTube notes"]
  node_youtube["Transcript loader<br/>[youtube_loader.py]"]
  node_chunker["Transcript chunker<br/>[youtube_chunker.py]"]
  node_notes["Notes generation<br/>[youtube_notes.py]"]
  node_export["Notes export<br/>[youtube_export.py]"]
end

subgraph group_extras["Assistant features"]
  node_ocr["Screenshot OCR<br/>[app.py]"]
  node_diagram["Mermaid generation"]
  node_tts["Text-to-speech<br/>[app.py]"]
end

node_user(("User"))
node_llm{{"Groq Llama"}}
node_ytapi{{"YouTube Transcript API"}}
node_tesseract{{"Tesseract OCR"}}
node_mermaid{{"Mermaid renderer"}}
node_gtts{{"gTTS"}}

node_user -->|"uses"| node_app
node_app -->|"loads and saves"| node_history
node_history -->|"reads and writes"| node_historyfile
node_app -->|"loads documents"| node_loader
node_loader -->|"returns documents"| node_app
node_app -->|"builds index"| node_store
node_store -->|"chunks and embeds"| node_embed
node_app -->|"submits query"| node_search
node_search -->|"checks CSVs"| node_csv
node_search -->|"queries vectors"| node_store
node_store -->|"returns matches"| node_search
node_search -->|"requests answer"| node_llm
node_llm -->|"returns answer"| node_search
node_search -->|"returns response"| node_app
node_user -->|"provides video URL"| node_app
node_app -->|"requests transcript"| node_youtube
node_youtube -->|"fetches transcript"| node_ytapi
node_youtube -->|"returns transcript"| node_app
node_app -->|"splits transcript"| node_chunker
node_app -->|"summarizes chunks"| node_notes
node_notes -->|"requests summaries"| node_llm
node_app -->|"generates final notes"| node_notes
node_app -->|"exports notes"| node_export
node_user -->|"provides screenshot"| node_app
node_app -->|"extracts text"| node_tesseract
node_app -->|"requests flowchart"| node_diagram
node_diagram -->|"generates Mermaid"| node_llm
node_app -->|"renders diagram"| node_mermaid
node_app -->|"reads response aloud"| node_tts
node_tts -->|"generates audio"| node_gtts

click node_app "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py"
click node_history "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/chat_history.py"
click node_historyfile "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/chat_history.json"
click node_loader "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/data_loader.py"
click node_embed "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/embedding.py"
click node_store "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/vectorestore.py"
click node_search "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/search.py"
click node_csv "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/csv_agent.py"
click node_youtube "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_loader.py"
click node_chunker "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_chunker.py"
click node_notes "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_notes.py"
click node_export "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_export.py"
click node_ocr "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py"
click node_diagram "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/diagram_generator.py"
click node_tts "https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_history,node_historyfile,node_user toneBlue
class node_loader,node_embed,node_store,node_search,node_csv toneAmber
class node_youtube,node_chunker,node_notes,node_export,node_ytapi toneMint
class node_ocr,node_diagram,node_tts toneRose
class node_llm,node_tesseract,node_mermaid,node_gtts toneIndigo
```

## Components

### User interface

- [Streamlit app](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py): `app.py`
- [Chat history](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/chat_history.py): `src/chat_history.py`
- [Conversation JSON](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/chat_history.json): `chat_history.json`

### Document RAG

- [File loader](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/data_loader.py): `src/data_loader.py`
- [Chunk and embed](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/embedding.py): `src/embedding.py`
- [FAISS store](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/vectorestore.py): `src/vectorestore.py`
- [RAG search](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/search.py): `src/search.py`
- [CSV analysis](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/csv_agent.py): `src/csv_agent.py`

### YouTube notes

- [Transcript loader](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_loader.py): `src/youtube_loader.py`
- [Transcript chunker](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_chunker.py): `src/youtube_chunker.py`
- [Notes generation](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_notes.py): `src/youtube_notes.py`
- [Notes export](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/youtube_export.py): `src/youtube_export.py`

### Assistant features

- [Screenshot OCR](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py): `app.py`
- [Mermaid generation](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/src/diagram_generator.py): `src/diagram_generator.py`
- [Text-to-speech](https://github.com/pritam-parali/document-parsing-using-rag-major-project/blob/main/app.py): `app.py`

### Other components

- **User**
- **Groq Llama**
- **YouTube Transcript API**
- **Tesseract OCR**
- **Mermaid renderer**
- **gTTS**

## Connections

- User → Streamlit app: uses
- Streamlit app → Chat history: loads and saves
- Chat history → Conversation JSON: reads and writes
- Streamlit app → File loader: loads documents
- File loader → Streamlit app: returns documents
- Streamlit app → FAISS store: builds index
- FAISS store → Chunk and embed: chunks and embeds
- Streamlit app → RAG search: submits query
- RAG search → CSV analysis: checks CSVs
- RAG search → FAISS store: queries vectors
- FAISS store → RAG search: returns matches
- RAG search → Groq Llama: requests answer
- Groq Llama → RAG search: returns answer
- RAG search → Streamlit app: returns response
- User → Streamlit app: provides video URL
- Streamlit app → Transcript loader: requests transcript
- Transcript loader → YouTube Transcript API: fetches transcript
- Transcript loader → Streamlit app: returns transcript
- Streamlit app → Transcript chunker: splits transcript
- Streamlit app → Notes generation: summarizes chunks
- Notes generation → Groq Llama: requests summaries
- Streamlit app → Notes generation: generates final notes
- Streamlit app → Notes export: exports notes
- User → Streamlit app: provides screenshot
- Streamlit app → Tesseract OCR: extracts text
- Streamlit app → Mermaid generation: requests flowchart
- Mermaid generation → Groq Llama: generates Mermaid
- Streamlit app → Mermaid renderer: renders diagram
- Streamlit app → Text-to-speech: reads response aloud
- Text-to-speech → gTTS: generates audio