# privateai-com/research-ai architecture

> Architecture diagram of the GitHub repository privateai-com/research-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/privateai-com/research-ai
- Repository: https://github.com/privateai-com/research-ai

## Overview

##### Project type and purpose
- This is a multi-component research assistant system (“research-ai”) that consists of three cooperating services:
- A Telegram Bot interface (bot/)
- A background Agent service (agent/)
- A REST API service (api/)
- Shared code (shared/) provides the database layer, LLM config, logging, and utilities.
- It implements a queue-based, multi-stage research pipeline that queries external sources (arXiv, Google Scholar, PubMed, GitHub), ranks and analyzes results with an LLM, then notifies users.

##### File structure
- Top-level directories:
- agent/ – orchestrator (manager.py) + pipeline stages (pipeline/) + source tools (browsing/)
- bot/ – Telegram handlers, notification logic, main entrypoint (main.py)
- api/ – FastAPI app (app.py) exposing REST endpoints
- shared/ – DB connection & operations, LLM config, event system, logging
- tests/ – automated tests for API, pipeline, and sources
- Start-scripts: start_bot.py, start_agent.py, start_api.py
- Config/build files: .env.example, pyproject.toml, quality-check.py

##### Readme insights
- Task flow: Bot/API → TaskQueue → Agent Manager → Pipeline → Sources & LLM → DB updates → Bot/API notifications
- Database: SQLite via SQLAlchemy, core models (User, TaskQueue, TaskStatistics, RateLimit), agent models (ResearchTopic, ArxivPaper, PaperAnalysis, UserSettings)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Interface Layer
    subgraph "User Interface" 
        direction TB
        User["User"]:::external
        Bot["Telegram Bot Service"]:::userFacing
        API["REST API Service"]:::userFacing
    end

    %% Common Layer (DB and Shared Utilities)
    subgraph "Common Layer" 
        direction TB
        DB[(TaskQueue & Database)]:::database
        SharedLLM["LLM Configuration & Abstraction"]:::shared
        EventSys["Event System (Notifications)"]:::shared
        Logging["Logging Utilities"]:::shared
    end

    %% Agent Layer
    subgraph "Agent Manager Service" 
        direction TB
        AgentMgr["Agent Manager"]:::engine
        subgraph "Research Pipeline"
            direction TB
            Pipeline["Pipeline"]:::pipeline
            Search["search()"]:::pipeline
            Ranking["ranking()"]:::pipeline
            Analyze["analyze()"]:::pipeline
            Formatting["formatting()"]:::pipeline
            Decision["decision()"]:::pipeline
            Strategy["strategy()"]:::pipeline
        end
        subgraph "Browsing Tools (Manual Sources)"
            direction TB
            Manual["Manual Sources"]:::browsing
            BaseSrc["base.py"]:::browsing
            GithubSrc["github.py"]:::browsing
            GoogleSrc["google_scholar.py"]:::browsing
            PubmedSrc["pubmed.py"]:::browsing
        end
    end

    %% External Dependencies
    subgraph "External Services"
        direction TB
        LLM["LLM Provider"]:::external
        Arxiv["arXiv API"]:::external
        GS["Google Scholar API"]:::external
        PM["PubMed API"]:::external
        GH["GitHub API"]:::external
    end

    %% Relationships
    User -->|"interacts"| Bot
    User -->|"interacts"| API

    Bot -->|"enqueueTask()"| DB
    API -->|"enqueueTask()"| DB

    DB -->|"pollQueue()"| AgentMgr
    AgentMgr -->|"invokePipeline()"| Pipeline

    Pipeline -->|"callSearch()"| Manual
    Manual -->|"useSources()"| Arxiv
    Manual -->|"useSources()"| GS
    Manual -->|"useSources()"| PM
    Manual -->|"useSources()"| GH

    Pipeline -->|"applyBM25"| Ranking
    Pipeline -->|"callLLM()"| Analyze
    Analyze -->|"uses"| LLM
    Analyze -->|"then"| Formatting
    Formatting --> Decision
    Decision --> Strategy

    Strategy -->|"results"| AgentMgr
    AgentMgr -->|"updateStatus()"| DB
    AgentMgr -->|"sendNotification()"| EventSys
    EventSys -->|"notify"| Bot
    EventSys -->|"notify"| API

    %% Click Events
    click Bot "https://github.com/privateai-com/research-ai/blob/master/bot/main.py"
    click Bot "https://github.com/privateai-com/research-ai/tree/master/bot/handlers/"
    click Bot "https://github.com/privateai-com/research-ai/blob/master/bot/utils.py"
    click API "https://github.com/privateai-com/research-ai/blob/master/api/app.py"
    click DB "https://github.com/privateai-com/research-ai/blob/master/shared/database/connection.py"
    click DB "https://github.com/privateai-com/research-ai/blob/master/shared/database/models.py"
    click DB "https://github.com/privateai-com/research-ai/blob/master/shared/database/operations/queue.py"
    click DB "https://github.com/privateai-com/research-ai/blob/master/shared/database/operations/task.py"
    click DB "https://github.com/privateai-com/research-ai/blob/master/shared/db.py"
    click AgentMgr "https://github.com/privateai-com/research-ai/blob/master/agent/manager.py"
    click Pipeline "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/pipeline.py"
    click Search "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/search.py"
    click Ranking "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/ranking.py"
    click Analyze "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/analyze.py"
    click Formatting "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/formatting.py"
    click Decision "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/decision.py"
    click Strategy "https://github.com/privateai-com/research-ai/blob/master/agent/pipeline/strategy.py"
    click Manual "https://github.com/privateai-com/research-ai/blob/master/agent/browsing/manual/manual.py"
    click BaseSrc "https://github.com/privateai-com/research-ai/blob/master/agent/browsing/manual/sources/base.py"
    click GithubSrc "https://github.com/privateai-com/research-ai/blob/master/agent/browsing/manual/sources/github.py"
    click GoogleSrc "https://github.com/privateai-com/research-ai/blob/master/agent/browsing/manual/sources/google_scholar.py"
    click PubmedSrc "https://github.com/privateai-com/research-ai/blob/master/agent/browsing/manual/sources/pubmed.py"
    click SharedLLM "https://github.com/privateai-com/research-ai/blob/master/shared/llm.py"
    click EventSys "https://github.com/privateai-com/research-ai/blob/master/shared/event_system.py"
    click Logging "https://github.com/privateai-com/research-ai/blob/master/shared/logging.py"

    %% Styles
    classDef userFacing fill:#B3CDE3,stroke:#333,stroke-width:1px
    classDef engine fill:#C7E9B4,stroke:#333,stroke-width:1px
    classDef pipeline fill:#CCEBC5,stroke:#333,stroke-width:1px
    classDef browsing fill:#E5D8BD,stroke:#333,stroke-width:1px
    classDef shared fill:#FDD0A2,stroke:#333,stroke-width:1px
    classDef database fill:#FDAE6B,stroke:#333,stroke-width:1px
    classDef external fill:#D9D9D9,stroke:#333,stroke-width:1px
```
