# khanz9664/personal_research_assistant architecture

> Architecture diagram of the GitHub repository khanz9664/personal_research_assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khanz9664/personal_research_assistant
- Repository: https://github.com/khanz9664/personal_research_assistant

## Overview

Step 1: Identify the Project Type and Purpose
- This is a Python-based “Personal Research Assistant.”
- It behaves like a full-stack (CLI/GUI front end + back-end logic) application that automates data retrieval (APIs, scraping), decision making (agent), orchestration (workflow), and report generation.
- Key use case: researchers interact via the UI, trigger a coordinated pipeline that fetches data, reasons about it, and outputs summaries/reports.

Step 2: Analyze the File Structure
- Top-level directories:
- ui/         – user interface (app.py)
- orchestrator/ – workflow coordinator
- agent/      – decision-making, planning, memory, tool registry
- data_access/ – modules for external data sources (search APIs, web scraping, LLM summarizer)
- output/     – report/summarization generator
- agent/tools/search contains sub-modules integrating with ArXiv, GitHub, HuggingFace Hub, PapersWithCode
- Requirements.txt and .env hint at external dependencies (HTTP libraries, LLM SDKs, scraping tools)

Step 3: Examine the README for Additional Insights
- Confirms modules: Agent, Data Access, Orchestrator, Output, UI
- No explicit database: “memory” is in-memory components (working vs. long-term)

Step 4: Main Components and Their Interactions
a. Components:
1. UI Layer (ui/app.py)
2. Orchestrator (orchestrator/orchestrator.py)
3. Agent Core
– Decomposer, DecisionMaker, Planner
– Memory: WorkingMemory & LongTermMemory
– Tool registry & search tools
– Evaluation (source_evaluator)
4. Data Access
– SearchAPI, WebScraper, LLMSummarizer
5. Output
– ReportGenerator
6. External Services
– ArXiv API, GitHub API, HuggingFace Hub, PapersWithCode, LLM provider, target websites

b. Relationships & Data Flows:
1. User → UI
2. UI → Orchestrator
3. Orchestrator invokes Agent
4. Agent decomposes user request → Planner → DecisionMaker
5. DecisionMaker calls Data Access or Agent.Tools
6. Data Access ↔ External Services
7. Fetched data → Agent.Evaluation → Agent.Memory
8. Orchestrator collects Agent outputs → Output.ReportGenerator
9. Report → UI → User

c. Architectural Patterns & Principles:
– Layered architecture: UI / Orchestrator / Agent / DataAccess / Output
– Single Responsibility: each module handles one concern
– Mediator (Orchestrator) coordinates loosely coupled modules
– Plugin pattern for search tools (registry + individual search modules)
– In-memory state management (working vs. long-term memory)

d. Technologies & Frameworks:
– Python 3.x
– HTTP clients (e.g., requests, aiohttp) for APIs and scraping
– LLM SDK (OpenAI, HuggingFace transformers) for summarization
– Possibly Flask or Typer for UI (if CLI/HTTP)
– Data serialization (JSON)
– Logging/Config via .env

- Treat ui/ as the “frontend” layer (even if CLI/web)
- Orchestrator + agent + data_access + output as backend services
- Workflow sequence: UI → Orchestrator → Agent → DataAccess → External → Agent → Output → UI

Step 6: Diagramming Guidelines
##### Use clear labels for each module (e.g., “Agent: Decomposer, Planner, DecisionMaker”).

Step 7: Emphasize Detail
- Every folder as a sub-component if space allows
- Tool registry pattern under Agent
- Two memory types (working vs. long-term)
- Don’t omit evaluation step (source_evaluator)
- Flow of data from External Services back into Agent memory and then onward to report generation

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "UI Layer"
        UI["UI Layer\n(ui/app.py)"]:::frontend
    end

    %% Backend Layers
    subgraph "Backend Services"
        direction TB

        Orch["Orchestrator\n(orchestrator/orchestrator.py)"]:::backend

        subgraph "Agent Core"
            direction TB
            Decomp["Decomposer\n(agent/decomposer.py)"]:::backend
            Plan["Planner\n(agent/planner.py)"]:::backend
            Decide["DecisionMaker\n(agent/decision_maker.py)"]:::backend

            subgraph "Agent Memory"
                direction TB
                WM[(WorkingMemory)]:::datastore
                LTM[(LongTermMemory)]:::datastore
            end

            Eval["SourceEvaluator\n(agent/evaluation/source_s_evaluator.py)"]:::backend
            Registry["Tool Registry\n(agent/tools/registry.py)"]:::backend

            subgraph "Search Tools"
                direction TB
                Arxiv["ArXiv Tool\n(agent/tools/search/arxiv.py)"]:::backend
                GitHub["GitHub Tool\n(agent/tools/search/github.py)"]:::backend
                HFHub["HuggingFace Hub Tool\n(agent/tools/search/huggingface_hub.py)"]:::backend
                PWC["PapersWithCode Tool\n(agent/tools/search/papers_with_code.py)"]:::backend
            end
        end

        subgraph "Data Access"
            direction TB
            SearchAPI["Search API\n(data_access/search_api.py)"]:::backend
            WebScraper["Web Scraper\n(data_access/web_scraper.py)"]:::backend
            LLMSum["LLM Summarizer\n(data_access/llm_summarizer.py)"]:::backend
        end

        subgraph "Output"
            Report["ReportGenerator\n(output/report_generator.py)"]:::backend
        end
    end

    %% External Services
    subgraph "External Services"
        direction TB
        ArxivAPI(["ArXiv API"]):::external
        GitHubAPI(["GitHub API"]):::external
        HFHubAPI(["HuggingFace Hub"]):::external
        PWCAPI(["PapersWithCode"]):::external
        LLMProv(["LLM Provider"]):::external
        Websites(["Target Websites"]):::external
    end

    %% Connections
    UI -->|"1. task request"| Orch
    Orch -->|"2. invoke agent pipeline"| Decomp
    Decomp -->|"3. plan tasks"| Plan
    Plan -->|"4. decide actions"| Decide
    Decide -->|"5. registry lookup"| Registry
    Registry -->|"6. select tool"| Arxiv
    Registry -->|"6. select tool"| GitHub
    Registry -->|"6. select tool"| HFHub
    Registry -->|"6. select tool"| PWC

    %% Data Access via DecisionMaker
    Decide -->|"7. call SearchAPI"| SearchAPI
    Decide -->|"7. call WebScraper"| WebScraper
    Decide -->|"7. call LLMSummarizer"| LLMSum

    %% External interactions
    SearchAPI -->|"8. fetch data"| ArxivAPI
    SearchAPI -->|"8. fetch data"| GitHubAPI
    SearchAPI -->|"8. fetch data"| HFHubAPI
    SearchAPI -->|"8. fetch data"| PWCAPI
    WebScraper -->|"8. scrape"| Websites
    LLMSum -->|"8. summarize via API"| LLMProv

    %% Data back to Agent
    ArxivAPI -->|"9. results"| Decide
    GitHubAPI -->|"9. results"| Decide
    HFHubAPI -->|"9. results"| Decide
    PWCAPI -->|"9. results"| Decide
    Websites -->|"9. scraped data"| Decide
    LLMProv -->|"9. summary"| Decide

    Decide -->|"10. evaluate sources"| Eval
    Eval -->|"11. store in memory"| WM
    Eval -->|"11. store long-term"| LTM

    %% Orchestration to Output
    Decide -->|"12. pass results"| Orch
    Orch -->|"13. generate report"| Report
    Report -->|"14. display report"| UI

    %% Click Events
    click UI "https://github.com/khanz9664/personal_research_assistant/blob/main/ui/app.py"
    click Orch "https://github.com/khanz9664/personal_research_assistant/blob/main/orchestrator/orchestrator.py"
    click Decomp "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/decomposer.py"
    click Plan "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/planner.py"
    click Decide "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/decision_maker.py"
    click WM "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/memory/working_memory.py"
    click LTM "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/memory/long_term_memory.py"
    click Eval "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/evaluation/source_s_evaluator.py"
    click Registry "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/tools/registry.py"
    click Arxiv "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/tools/search/arxiv.py"
    click GitHub "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/tools/search/github.py"
    click HFHub "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/tools/search/huggingface_hub.py"
    click PWC "https://github.com/khanz9664/personal_research_assistant/blob/main/agent/tools/search/papers_with_code.py"
    click SearchAPI "https://github.com/khanz9664/personal_research_assistant/blob/main/data_access/search_api.py"
    click WebScraper "https://github.com/khanz9664/personal_research_assistant/blob/main/data_access/web_scraper.py"
    click LLMSum "https://github.com/khanz9664/personal_research_assistant/blob/main/data_access/llm_summarizer.py"
    click Report "https://github.com/khanz9664/personal_research_assistant/blob/main/output/report_generator.py"

    %% Styles
    classDef frontend fill:#E8F0FE,stroke:#4285F4,stroke-width:2px;
    classDef backend fill:#F1F8E9,stroke:#8BC34A,stroke-width:2px;
    classDef datastore fill:#FFF8E1,stroke:#FFEB3B,stroke-width:2px,stroke-dasharray: 5 5;
    classDef external fill:#FCE4EC,stroke:#E91E63,stroke-width:2px,stroke-dasharray: 2 2;
```
