# epuerta9/deep-research-py architecture

> Architecture diagram of the GitHub repository epuerta9/deep-research-py, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/epuerta9/deep-research-py
- Repository: https://github.com/epuerta9/deep-research-py

## Overview

##### Main Components
- User Interface (CLI): Represented by the main entry point (run.py) which gathers research topics, research breadth, depth, and follow-up questions.
- Core Research Logic: The “deep_research.py” module serves as the central brain, coordinating research activities.
- AI Services: The “ai” directory contains modules like providers.py (configuring the AI service clients) and text_splitter.py (for text preprocessing).
- Data Acquisition: The “data_acquisition” folder holds modules such as manager.py, scraper.py, search.py, and services.py that handle web searches and scraping.
- Prompt & Feedback Handling: Modules like prompt.py (for composing system-prompts) and feedback.py (for follow-up question generation).
- Configuration Module: The config.py and .env file indicate where API keys and runtime settings are managed.
- Deployment & Containerization: Dockerfile and docker-compose.yml illustrate that the application can be containerized and integrated with external tools (e.g., OpenWebUI).

##### Relationships & Interactions
- The CLI (run.py) gathers user inputs and triggers the core research process.
- The Core Research module (deep_research.py) orchestrates calling out to:
- The Data Acquisition layer (web search & scraping via data_acquisition/ modules) to fetch relevant source content.
- The AI Services layer (modules under ai/) which processes inputs and generates content based on prompts provided by prompt.py.
- The Feedback module that interacts with the user to refine the search or expand upon findings.
- There is a clear separation whereby configuration (config.py and .env) feeds parameters such as API keys into both the Data Acquisition and AI Services layers.
- Optionally, when deployed with Docker/OpenWebUI, an external UI layer is placed in front of the CLI, routing requests to the back-end API (configured via docker-compose and exposed endpoints).

##### Architectural Patterns and Technologies
- Separation of Concerns: The project cleanly separates its logic into distinct modules responsible for UI, research logic, data fetching, AI interfacing, and configuration.
- CLI-Driven Workflow: The primary user interaction is command-line oriented, with well-defined modules handling distinct research functionalities.
- Use of External APIs: The AI (OpenAI or DeepSeek) and web search (Firecrawl) integrations are configurable, implying the design emphasizes flexibility and extensibility.
- Containerization: Docker and docker-compose usage indicates a micro-deployment container model to facilitate integration with tools like OpenWebUI.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Layer
    CLI["User / CLI Interface (run.py)"]:::cli

    %% Core Research Module
    CORE["Core Research Module (deep_research.py)"]:::core
    PROMPT["Prompt Generator (prompt.py)"]:::core
    FEEDBACK["Feedback Handler (feedback.py)"]:::core
    CORE -->|"calls"| PROMPT
    CORE -->|"calls"| FEEDBACK

    %% Support Layers
    DATA["Data Acquisition Layer (data_acquisition)"]:::data
    AI["AI Service Layer (ai)"]:::ai
    PROV["Providers & Configuration (providers.py)"]:::ai
    TEXT["Text Processing (text_splitter.py)"]:::ai
    AI -->|"uses"| PROV
    AI -->|"uses"| TEXT

    %% Configuration
    CONFIG["Configuration (config.py & .env)"]:::config

    %% Deployment & External Services
    subgraph "Deployment & External Services"
        DOCKERFILE["Dockerfile"]:::deploy
        DOCKERCOMPOSE["docker-compose.yml"]:::deploy
        EXT_UI["External UI (OpenWebUI)"]:::external
        EXT_API_AI["External AI (OpenAI/DeepSeek)"]:::external
        EXT_API_FC["External Web Scraping (Firecrawl)"]:::external
    end

    %% Data Flow Connections
    CLI -->|"triggers"| CORE
    CORE -->|"fetches"| DATA
    CORE -->|"processes"| AI
    CONFIG ---|"configures"| CORE
    CONFIG ---|"configures"| DATA
    CONFIG ---|"configures"| AI
    DOCKERFILE --- DOCKERCOMPOSE
    DOCKERFILE ---|"containerizes"| CORE
    DOCKERFILE ---|"connects"| EXT_UI
    DOCKERFILE ---|"integrates"| EXT_API_AI
    DOCKERFILE ---|"integrates"| EXT_API_FC

    %% Styles
    classDef cli fill:#AED6F1,stroke:#2E86C1,stroke-width:2px;
    classDef core fill:#A3E4D7,stroke:#1E8449,stroke-width:2px;
    classDef data fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef ai fill:#F5CBA7,stroke:#D35400,stroke-width:2px;
    classDef config fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px;
    classDef deploy fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef external fill:#E8DAEF,stroke:#6C3483,stroke-width:2px;

    %% Click Events
    click CLI "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/run.py"
    click CORE "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/deep_research.py"
    click PROMPT "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/prompt.py"
    click FEEDBACK "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/feedback.py"
    click PROV "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/ai/providers.py"
    click TEXT "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/ai/text_splitter.py"
    click DATA "https://github.com/epuerta9/deep-research-py/tree/main/deep_research_py/data_acquisition/"
    click CONFIG "https://github.com/epuerta9/deep-research-py/blob/main/deep_research_py/config.py"
    click DOCKERFILE "https://github.com/epuerta9/deep-research-py/tree/main/Dockerfile"
    click DOCKERCOMPOSE "https://github.com/epuerta9/deep-research-py/blob/main/docker-compose.yml"
```
