# poojithdevan4d/munshi architecture

> Architecture diagram of the GitHub repository poojithdevan4d/munshi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/poojithdevan4d/munshi
- Repository: https://github.com/poojithdevan4d/munshi

## Overview

Munshi is an on-premise assistant for CA firms. Staff ask questions through the Streamlit UI and receive answers grounded in client PDFs, with source citations. The main document workflow detects typed versus scanned PDFs, extracts text or OCR, indexes client-tagged content in Qdrant, then retrieves context for local LLM answer generation. The UI also supports live PDF uploads. The graph includes both indexed ingestion and documented query capabilities; several UI details and the local model server are described in the README but not fully shown in sampled code.

Main entry points: [`data/sharma_associates`](https://github.com/poojithdevan4d/munshi/tree/main/data/sharma_associates) (Client PDFs), [`app/munshi_ui.py`](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py) (RAG query engine), [`code/ingest_all_clients.py`](https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py) (Chunk and embed), [`code/query_full_firm.py`](https://github.com/poojithdevan4d/munshi/blob/main/code/query_full_firm.py) (Firm query CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Staff interface"]
  node_ui["Streamlit UI<br/>[munshi_ui.py]"]
  node_legacy_ui["Earlier UI"]
  node_upload["PDF upload<br/>[munshi_ui.py]"]
  node_chat["Chat and citations<br/>[munshi_ui.py]"]
  node_answer["Cited answer<br/>[munshi_ui.py]"]
end

subgraph group_documents["Document processing"]
  node_typed_extract["Typed PDF extraction"]
  node_ocr["Scanned PDF OCR"]
  node_indexing["Chunk and embed"]
end

subgraph group_retrieval["Retrieval and answers"]
  node_embedding["Local embeddings"]
  node_qdrant[("Qdrant index")]
  node_query["RAG query engine<br/>[munshi_ui.py]"]
  node_llm["Local LLM server"]
  node_firm_query["Firm query CLI<br/>[query_full_firm.py]"]
  node_quick_query["Initial RAG demo<br/>[rag_first_query.py]"]
end

subgraph group_data["Client data"]
  node_client_docs["Client PDFs"]
  node_dataset_full["Synthetic firm data"]
  node_dataset_acme["Acme sample data"]
end

node_staff(("CA staff"))

node_staff -->|"uses"| node_ui
node_ui -->|"offers chat"| node_chat
node_staff -->|"uploads PDFs"| node_upload
node_upload -->|"saves PDFs"| node_client_docs
node_client_docs -->|"reads PDFs"| node_typed_extract
node_typed_extract -->|"extracts text"| node_indexing
node_client_docs -->|"reads scanned PDFs"| node_ocr
node_ocr -->|"supplies OCR text"| node_indexing
node_indexing -->|"embeds chunks"| node_embedding
node_indexing -->|"stores vectors"| node_qdrant
node_chat -->|"submits questions"| node_query
node_query -->|"retrieves context"| node_qdrant
node_query -->|"generates answer"| node_llm
node_query -->|"returns sources"| node_answer
node_answer -->|"renders citations"| node_chat
node_firm_query -->|"queries index"| node_qdrant
node_firm_query -->|"requests generation"| node_llm
node_quick_query -->|"indexes and queries"| node_qdrant
node_quick_query -->|"requests generation"| node_llm
node_dataset_full -.->|"generates documents"| node_client_docs
node_dataset_acme -.->|"generates documents"| node_client_docs

click node_ui "https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py"
click node_legacy_ui "https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui_v1_complete.py"
click node_upload "https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py"
click node_chat "https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py"
click node_client_docs "https://github.com/poojithdevan4d/munshi/tree/main/data/sharma_associates"
click node_typed_extract "https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py"
click node_ocr "https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py"
click node_indexing "https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py"
click node_embedding "https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py"
click node_query "https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py"
click node_answer "https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py"
click node_firm_query "https://github.com/poojithdevan4d/munshi/blob/main/code/query_full_firm.py"
click node_quick_query "https://github.com/poojithdevan4d/munshi/blob/main/code/rag_first_query.py"
click node_dataset_full "https://github.com/poojithdevan4d/munshi/blob/main/code/generate_full_dataset.py"
click node_dataset_acme "https://github.com/poojithdevan4d/munshi/blob/main/code/generate_data_part1.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui,node_legacy_ui,node_upload,node_chat,node_answer toneBlue
class node_typed_extract,node_ocr,node_indexing toneAmber
class node_embedding,node_qdrant,node_query,node_llm,node_firm_query,node_quick_query toneMint
class node_client_docs,node_dataset_full,node_dataset_acme toneRose
class node_staff toneIndigo
```

## Components

### Staff interface

- [Streamlit UI](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py): `app/munshi_ui.py`
- [Earlier UI](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui_v1_complete.py): `app/munshi_ui_v1_complete.py`
- [PDF upload](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py): `app/munshi_ui.py`
- [Chat and citations](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py): `app/munshi_ui.py`
- [Cited answer](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py): `app/munshi_ui.py`

### Document processing

- [Typed PDF extraction](https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py): `code/ingest_all_clients.py`
- [Scanned PDF OCR](https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py): `code/ingest_all_clients.py`
- [Chunk and embed](https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py): `code/ingest_all_clients.py`

### Retrieval and answers

- [Local embeddings](https://github.com/poojithdevan4d/munshi/blob/main/code/ingest_all_clients.py): `code/ingest_all_clients.py`
- **Qdrant index**
- [RAG query engine](https://github.com/poojithdevan4d/munshi/blob/main/app/munshi_ui.py): `app/munshi_ui.py`
- **Local LLM server**
- [Firm query CLI](https://github.com/poojithdevan4d/munshi/blob/main/code/query_full_firm.py): `code/query_full_firm.py`
- [Initial RAG demo](https://github.com/poojithdevan4d/munshi/blob/main/code/rag_first_query.py): `code/rag_first_query.py`

### Client data

- [Client PDFs](https://github.com/poojithdevan4d/munshi/tree/main/data/sharma_associates): `data/sharma_associates`
- [Synthetic firm data](https://github.com/poojithdevan4d/munshi/blob/main/code/generate_full_dataset.py): `code/generate_full_dataset.py`
- [Acme sample data](https://github.com/poojithdevan4d/munshi/blob/main/code/generate_data_part1.py): `code/generate_data_part1.py`

### Other components

- **CA staff**

## Connections

- CA staff → Streamlit UI: uses (evidence: `README.md`)
- Streamlit UI → Chat and citations: offers chat (evidence: `README.md`)
- CA staff → PDF upload: uploads PDFs (evidence: `README.md`)
- PDF upload → Client PDFs: saves PDFs (evidence: `app/munshi_ui.py`)
- Client PDFs → Typed PDF extraction: reads PDFs (evidence: `code/ingest_all_clients.py`)
- Typed PDF extraction → Chunk and embed: extracts text (evidence: `code/ingest_all_clients.py`)
- Client PDFs → Scanned PDF OCR: reads scanned PDFs (evidence: `code/ingest_all_clients.py`)
- Scanned PDF OCR → Chunk and embed: supplies OCR text (evidence: `code/ingest_all_clients.py`)
- Chunk and embed → Local embeddings: embeds chunks (evidence: `code/ingest_all_clients.py`)
- Chunk and embed → Qdrant index: stores vectors (evidence: `code/ingest_all_clients.py`)
- Chat and citations → RAG query engine: submits questions (evidence: `README.md`)
- RAG query engine → Qdrant index: retrieves context (evidence: `README.md`)
- RAG query engine → Local LLM server: generates answer (evidence: `README.md`)
- RAG query engine → Cited answer: returns sources (evidence: `app/munshi_ui.py`)
- Cited answer → Chat and citations: renders citations (evidence: `app/munshi_ui.py`)
- Firm query CLI → Qdrant index: queries index (evidence: `code/query_full_firm.py`)
- Firm query CLI → Local LLM server: requests generation (evidence: `code/query_full_firm.py`)
- Initial RAG demo → Qdrant index: indexes and queries (evidence: `code/rag_first_query.py`)
- Initial RAG demo → Local LLM server: requests generation (evidence: `code/rag_first_query.py`)
- Synthetic firm data → Client PDFs: generates documents
- Acme sample data → Client PDFs: generates documents