# ysnhdn/assistdoc architecture

> Architecture diagram of the GitHub repository ysnhdn/assistdoc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ysnhdn/assistdoc
- Repository: https://github.com/ysnhdn/assistdoc

## Overview

##### Project type and purpose
- AssistDoc is a full-stack, RAG-powered document assistant built with Streamlit. Its purpose is to let users upload PDF/DOCX/TXT files, automatically index and embed their contents, then interact via question-answering, summarization, or structured-data extraction using LLMs (GitHub Inference, Hugging Face, etc.) and a local vector store (FAISS/ChromaDB).

##### File structure
- Top-level:
- app.py – Streamlit entry point and router
- .streamlit/ – config & setup scripts
- scripts/index_documents.py – batch indexing utility
- Dockerfile, setup.py, requirements.txt
- data/ – JSON registry + persistent vector_store & uploaded_files (in README)
- src/:
- document_processor/ (parser → chunker → embedder)
- vector_db/ (store.py ↔ Faiss/ChromaDB, retriever.py)
- llm/ (models.py, prompts.py, chain.py)
- agents/ (high-level orchestrators: qa_agent, summary_agent, extraction_agent)
- utils/ (helpers, API/session utilities)
- ui/:
- pages/ (home, qa, summary, extraction)
- components/ (sidebar upload, visualization, document selector)
- models/: pre-downloaded sentence-transformers model files
- test/: unit tests for core modules

##### Key architecture insights from README
- Streamlit front end drives interaction
- Persistent storage of source docs + vector store between sessions
- Seamless switch between multiple LLM providers
- RAG pipeline: upload → parse → chunk → embed → store → retrieve → LLM chain

##### Main components
a. **User Interface (Streamlit)**
- app.py orchestrates pages & sidebar
- ui/pages and ui/components render controls and results
b. **Document Processing Pipeline**
- parser: extract text from PDF/DOCX/TXT
- chunker: split text into manageable chunks
- embedder: call sentence-transformers to generate embeddings
c. **Vector Database**
- store.py: create/manage FAISS or ChromaDB index on disk
- retriever.py: semantic search for top-K chunks
- data/documents_registry.json: metadata registry
d. **LLM Layer**
- chain.py: orchestrates prompt templates + retrieved context
- prompts.py: templates for QA, summarization, extraction
- models.py: config for providers (GitHub, OpenAI, etc.)
- External API: GitHub Inference / Hugging Face / OpenAI endpoints
e. **Agents**
- qa_agent, summary_agent, extraction_agent: high-level wrappers that call retriever + chain
f. **Persistent Data Store**
- data/uploaded_files – raw docs
- vector_store – serialized FAISS/ChromaDB index

##### Relationships & interactions
1. User uploads document via Streamlit UI →
2. document_processor.parser → chunker → embedder → vector_db.store → update registry
3. User selects action (QA/Summary/Extraction) → UI calls corresponding agent
4. Agent calls vector_db.retriever to fetch top-K embeddings + chunks
5. Agent calls llm.chain with chunks + prompt → External LLM API or local model
6. llm.chain returns result → UI displays with visualization components

##### Architectural patterns and design principles
- **Modular pipeline**: clear separation of parsing, chunking, embedding
- **Adapter pattern**: vector_db abstracts FAISS vs. ChromaDB, llm/models.py abstracts different providers
- **Agent orchestration**: high-level agents hide complexity from UI
- **Persistence layer**: registry JSON + on-disk vector store ensure stateful sessions

##### Elements
- **Clear labels**: e.g., “Parser”, “Chunker”, “Embedder”, “Agent: QA”
- Upload → Parser → Chunker → Embedder → Store
- UI → Agent → Retriever → LLM Chain → UI
- UI components (light blue rectangles)
- Processing modules (yellow rectangles)

##### Detailed capture of essential elements
- Location of config.py (API key injection)
- Pre-downloaded model directory under models/all-MiniLM-L6-v2
- Test suite separately (optional, for dev flow)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    subgraph "UI Layer"
        APP["Streamlit Entry Point (app.py)"]:::ui
        Home["Page: Home"]:::ui
        QAPage["Page: QA"]:::ui
        SummaryPage["Page: Summary"]:::ui
        ExtractionPage["Page: Extraction"]:::ui
        DocumentUploader["Component: Document Uploader"]:::ui
        Sidebar["Component: Sidebar"]:::ui
        Visualization["Component: Visualization"]:::ui
    end

    %% Agents Layer
    subgraph "Agents"
        QAAgent["Agent: QA"]:::agent
        SAgent["Agent: Summary"]:::agent
        EAgent["Agent: Extraction"]:::agent
    end

    %% Data Processing & Retrieval
    subgraph "Data Processing & Retrieval"
        Parser["Parser"]:::proc
        Chunker["Chunker"]:::proc
        Embedder["Embedder"]:::proc
        Store["Vector Store Adapter"]:::proc
        Retriever["Retriever"]:::proc
    end

    %% LLM Layer
    subgraph "LLM Layer"
        Chain["Chain Orchestration"]:::lllayer
        Prompts["Prompt Templates"]:::lllayer
        Models["Provider Configuration"]:::lllayer
    end

    %% External Services & Storage
    subgraph "External Services & Storage"
        Registry["Documents Registry"]:::storage
        VectorStore["Vector Store (FAISS/ChromaDB)"]:::storage
        FileSystem["Uploaded Files Store"]:::storage
        LLM_API["External LLM API"]:::ext
        ModelDir["Local Model Directory"]:::storage
    end

    %% UI interactions
    APP --> DocumentUploader
    APP --> Sidebar
    APP --> Home
    APP --> QAPage
    APP --> SummaryPage
    APP --> ExtractionPage

    %% Upload flow
    DocumentUploader --> Parser
    Parser --> Chunker
    Chunker --> Embedder
    Embedder --> Store
    Store --> Registry
    Store --> VectorStore
    Store --> FileSystem

    %% User action flow
    Home --> QAAgent
    QAPage --> QAAgent
    SummaryPage --> SAgent
    ExtractionPage --> EAgent

    %% Retrieval & LLM
    QAAgent --> Retriever
    SAgent --> Retriever
    EAgent --> Retriever
    Retriever --> Chain
    Chain --> Prompts
    Chain --> Models
    Chain --> LLM_API
    Chain --> ModelDir

    %% Results to UI
    Chain --> Visualization
    Visualization --> QAPage
    Visualization --> SummaryPage
    Visualization --> ExtractionPage

    %% Batch indexing utility
    IndexScript["Batch Indexing Utility"]:::proc
    IndexScript --> Parser

    %% Config & Utils
    StreamlitConfig["Streamlit Config"]:::storage
    NLTKSetup["NLTK Setup"]:::storage
    StreamlitConfig --> APP
    NLTKSetup --> APP

    Helpers["Utility Helpers"]:::proc
    APIHelpers["API Helpers"]:::proc
    SessionUtils["Session Utils"]:::proc
    Chain --> APIHelpers
    Chain --> SessionUtils
    Embedder --> Helpers

    %% Click Events
    click APP "https://github.com/ysnhdn/assistdoc/blob/main/app.py"
    click Home "https://github.com/ysnhdn/assistdoc/blob/main/ui/pages/home.py"
    click QAPage "https://github.com/ysnhdn/assistdoc/blob/main/ui/pages/qa.py"
    click SummaryPage "https://github.com/ysnhdn/assistdoc/blob/main/ui/pages/summary.py"
    click ExtractionPage "https://github.com/ysnhdn/assistdoc/blob/main/ui/pages/extraction.py"
    click DocumentUploader "https://github.com/ysnhdn/assistdoc/blob/main/ui/components/document_uploader.py"
    click Sidebar "https://github.com/ysnhdn/assistdoc/blob/main/ui/components/sidebar.py"
    click Visualization "https://github.com/ysnhdn/assistdoc/blob/main/ui/components/visualization.py"
    click Parser "https://github.com/ysnhdn/assistdoc/blob/main/src/document_processor/parser.py"
    click Chunker "https://github.com/ysnhdn/assistdoc/blob/main/src/document_processor/chunker.py"
    click Embedder "https://github.com/ysnhdn/assistdoc/blob/main/src/document_processor/embedder.py"
    click Store "https://github.com/ysnhdn/assistdoc/blob/main/src/vector_db/store.py"
    click Retriever "https://github.com/ysnhdn/assistdoc/blob/main/src/vector_db/retriever.py"
    click Registry "https://github.com/ysnhdn/assistdoc/blob/main/data/documents_registry.json"
    click Chain "https://github.com/ysnhdn/assistdoc/blob/main/src/llm/chain.py"
    click Prompts "https://github.com/ysnhdn/assistdoc/blob/main/src/llm/prompts.py"
    click Models "https://github.com/ysnhdn/assistdoc/blob/main/src/llm/models.py"
    click QAAgent "https://github.com/ysnhdn/assistdoc/blob/main/src/agents/qa_agent.py"
    click SAgent "https://github.com/ysnhdn/assistdoc/blob/main/src/agents/summary_agent.py"
    click EAgent "https://github.com/ysnhdn/assistdoc/blob/main/src/agents/extraction_agent.py"
    click IndexScript "https://github.com/ysnhdn/assistdoc/blob/main/scripts/index_documents.py"
    click StreamlitConfig "https://github.com/ysnhdn/assistdoc/blob/main/.streamlit/config.toml"
    click NLTKSetup "https://github.com/ysnhdn/assistdoc/blob/main/.streamlit/nltk-setup.py"
    click ModelDir "https://github.com/ysnhdn/assistdoc/tree/main/models/all-MiniLM-L6-v2/"
    click Helpers "https://github.com/ysnhdn/assistdoc/blob/main/src/utils/helpers.py"
    click APIHelpers "https://github.com/ysnhdn/assistdoc/blob/main/utils/api_helpers.py"
    click SessionUtils "https://github.com/ysnhdn/assistdoc/blob/main/utils/session_utils.py"

    %% Styles
    classDef ui fill:#E0F7FA,stroke:#006064
    classDef proc fill:#FFF9C4,stroke:#F57F17
    classDef agent fill:#C8E6C9,stroke:#2E7D32
    classDef lllayer fill:#D1C4E9,stroke:#512DA8
    classDef storage fill:#CFD8DC,stroke:#455A64
    classDef ext fill:#FFCDD2,stroke:#C62828
```
