# pranav1011/research_paper_assistant architecture

> Architecture diagram of the GitHub repository pranav1011/research_paper_assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pranav1011/research_paper_assistant
- Repository: https://github.com/pranav1011/research_paper_assistant

## Overview

##### Project Type and Purpose
- This is a Python-based Research Paper Assistant combining a web interface (Gradio) and a CLI pipeline.
- It ingests PDF research papers, parses and chunks them, builds a FAISS vector index, retrieves relevant passages, and uses an LLM to answer user questions.

a. Presentation Layer
- Web UI (app.py – Gradio server on localhost:7860)
- CLI Orchestrator (automated_pipeline.py / main.py)
b. Business Logic / Orchestration
- Orchestrator Module (automated_pipeline.py / main.py)
- Configuration Module (config/config.py)
- Backend Prompt Templates (backend/prompts/templates.py)
c. Data Processing Services
- PDF Parser (scripts/prepare_magic_pdf_chunks.py or a mineru-based parser)
- Chunker (scripts/chunking_md_sections.py / scripts/process_chunks.py)
- Embedding & Index Builder (vectorstore/faiss_index)
- Hybrid Retriever (vectorstore/hybrid_retriever.py)
d. External Services
- LLM API Client (scripts/llm_answer.py calling OpenAI/HuggingFace)
- Evaluation Scripts (scripts/llm_eval_metric.py, scripts/llm_evaluate_answers.py)
e. Data Stores
- Vector Store (FAISS indices on disk under vectorstore/faiss_index)
- Raw & Intermediate Data (data/papers, data/…chunks)
f. Support Modules
- Logging & Config (requirements.txt, config/)
- Tests (tests/…)

1. User → Web UI or CLI
2. UI/CLI → Orchestrator
3. Orchestrator
→ PDF Parser → Extracted Text → Chunker → Text Chunks
→ Embedding Service → Vectorstore Index (persisted on disk)
4. User Query → Hybrid Retriever → Candidate Chunks
5. Candidate Chunks + Prompt Templates → LLM API Client → Model Response
6. Response → UI/CLI → User
7. (Optional) Orchestrator → Evaluation Scripts → Metrics → Logs

##### Architectural Patterns & Design Principles
- Layered Architecture: Presentation (UI/CLI) → Orchestration → Services → Storage → External
- Modular Services: separate scripts for parsing, chunking, retrieval, evaluation
- Config-Driven: config/config.py centralizes environment settings
- Reusable Prompt Templates: backend/prompts/templates.py for LLM queries
- Test-Driven: tests/ directory covers each major component

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation Layer
    subgraph "Presentation Layer"
        UI["Web UI (Gradio)"]:::internal
        CLI["CLI Orchestrator"]:::internal
    end

    UI -->|"User Request"| Orch
    CLI -->|"CLI Trigger"| Orch

    %% Orchestration Layer
    subgraph "Orchestration"
        Orch["Orchestrator Module"]:::internal
        Config["Configuration Module"]:::internal
        Templates["Prompt Templates"]:::internal
    end

    Orch --> Config
    Orch --> Templates

    %% Data Processing Layer
    subgraph "Data Processing"
        Parser["PDF Parser Service"]:::internal
        Chunker["Chunker Service"]:::internal
        Embed["Embedding & Index Builder"]:::internal
        Retriever["Hybrid Retriever Service"]:::internal
    end

    Orch -->|"PDF file"| Parser
    Parser -->|"Extracted Text"| Chunker
    Chunker -->|"Text Chunks"| Embed
    Embed -->|"Embeddings"| VStore

    %% Data Stores
    subgraph "Data Stores"
        VStore(["FAISS Vector Store"]):::db
        RawData["Raw & Intermediate Data"]:::db
    end

    Parser --> RawData
    Chunker --> RawData
    Embed --> VStore

    %% Retrieval & LLM Flow
    UI -->|"User Query"| Retriever
    CLI -->|"User Query"| Retriever
    Retriever -->|"Candidate Chunks"| Templates
    Templates -->|"Prompt + Context"| LLM

    %% External Services
    subgraph "External Services"
        LLM["LLM API Client"]:::external
        Eval["Evaluation Scripts"]:::external
    end

    LLM -->|"Model Response"| Orch
    Orch -->|"Response"| UI
    Orch -->|"Response"| CLI

    Orch -->|"Run Evaluation"| Eval
    Eval -->|"Metrics"| RawData

    %% Click Events
    click UI "https://github.com/pranav1011/research_paper_assistant/blob/main/app.py"
    click CLI "https://github.com/pranav1011/research_paper_assistant/blob/main/automated_pipeline.py"
    click Orch "https://github.com/pranav1011/research_paper_assistant/blob/main/automated_pipeline.py"
    click Config "https://github.com/pranav1011/research_paper_assistant/blob/main/config/config.py"
    click Templates "https://github.com/pranav1011/research_paper_assistant/blob/main/backend/prompts/templates.py"
    click Parser "https://github.com/pranav1011/research_paper_assistant/blob/main/scripts/prepare_magic_pdf_chunks.py"
    click Chunker "https://github.com/pranav1011/research_paper_assistant/blob/main/scripts/chunking_md_sections.py"
    click Chunker "https://github.com/pranav1011/research_paper_assistant/blob/main/scripts/process_chunks.py"
    click VStore "https://github.com/pranav1011/research_paper_assistant/tree/main/vectorstore/faiss_index/"
    click Retriever "https://github.com/pranav1011/research_paper_assistant/blob/main/vectorstore/hybrid_retriever.py"
    click LLM "https://github.com/pranav1011/research_paper_assistant/blob/main/scripts/llm_answer.py"
    click Eval "https://github.com/pranav1011/research_paper_assistant/blob/main/scripts/llm_eval_metric.py"
    click Eval "https://github.com/pranav1011/research_paper_assistant/blob/main/scripts/llm_evaluate_answers.py"
    click RawData "https://github.com/pranav1011/research_paper_assistant/tree/main/data/papers/"

    %% Styles
    classDef internal fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef db fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#FFD580,stroke:#333,stroke-width:1px
```
