# qx-labs/agents-deep-research architecture

> Architecture diagram of the GitHub repository qx-labs/agents-deep-research, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/qx-labs/agents-deep-research
- Repository: https://github.com/qx-labs/agents-deep-research

## Overview

##### Project Type and Purpose
- This is an open-source Python library and CLI tool for “agentic deep research,” built on top of the OpenAI Agents SDK.
- It orchestrates multiple specialized agents (knowledge-gap, planner, tool-selector, tool-agents, writer, proofreader) to perform iterative or parallelized research and synthesize reports.

##### Top-Level Architecture
- User Interfaces:
- Python API (IterativeResearcher, DeepResearcher classes)
- CLI entry point (`deep_researcher/main.py`)
- Core Orchestrators:
- IterativeResearcher (single-topic loop)
- DeepResearcher (outline + parallel sub-research loops + final proofing)
- Agent Layer:
- PlannerAgent
- KnowledgeGapAgent
- ToolSelectorAgent
- ToolAgents (SearchAgent, CrawlAgent, etc.)
- Observations/ThinkingAgent
- WriterAgent
- ProofreaderAgent
- Tools Layer:
- Web search (`deep_researcher/tools/web_search.py`)
- Website crawler (`deep_researcher/tools/crawl_website.py`)
- Custom tools (extendable)
- Config & Utilities:
- LLMConfig (`deep_researcher/llm_config.py`)
- Parsing, markdown→PDF, OS utils
- External Systems:
- LLM providers (OpenAI, Azure, Anthropic, Gemini, etc.)
- Search providers (Serper, OpenAI tool calling)
- Optional .env for API keys

##### Component Relationships & Data Flows
- User input (CLI or API) → chosen Researcher (Iterative or Deep)
- DeepResearcher:
1. Calls PlannerAgent → generates report outline (sections)
2. Spawns multiple IterativeResearcher instances in parallel (one per section)
3. Each IterativeResearcher loops:
a. KnowledgeGapAgent analyzes current draft → gaps
b. ToolSelectorAgent chooses which ToolAgents to invoke
c. ToolAgents execute web searches or crawls → new findings
d. ObservationsAgent synthesizes findings → feeds back to KnowledgeGapAgent
e. After loop ends, WriterAgent drafts section
4. ProofreaderAgent combines section drafts → final report
- IterativeResearcher (stand-alone):
- Same inner loop (KnowledgeGap → Selector → Tools → Observations)
- Ends with WriterAgent output

##### Architectural Patterns & Design Principles
- Modular “agent” components: each agent encapsulates a single responsibility.
- Orchestration layer (IterativeResearcher / DeepResearcher) implements the control flow.
- Parallelism: DeepResearcher runs section-level researchers concurrently.
- Extension points: ToolAgents and custom tools can be dropped in without modifying core logic.
- Async I/O: Uses Python asyncio for parallel tool calls and model API requests.
- Configuration Abstraction: LLMConfig separates model-provider specifics from orchestration logic.

##### Technologies & Frameworks
- Python 3, asyncio
- OpenAI Agents SDK
- External LLM APIs (OpenAI, Azure, Anthropic, Gemini, etc.)
- Serper API (for Google search) or OpenAI tool calling
- Markdown & PDF utilities

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI / API Layer
    subgraph "UI / API" 
        direction TB
        User["User"]:::external
        PythonAPI["Python API\n(IterativeResearcher, DeepResearcher)"]:::frontend
        CLI["CLI Entry Point\n(main.py)"]:::frontend
    end

    %% Orchestration Layer
    subgraph "Orchestrators" 
        direction TB
        IterativeResearcher["IterativeResearcher"]:::orchestrator
        DeepResearcher["DeepResearcher"]:::orchestrator
    end

    %% Agent Layer
    subgraph "Agents" 
        direction TB
        PlannerAgent["PlannerAgent"]:::agent
        KnowledgeGapAgent["KnowledgeGapAgent"]:::agent
        ToolSelectorAgent["ToolSelectorAgent"]:::agent
        subgraph "ToolAgents"
            direction TB
            SearchAgent["SearchAgent"]:::agent
            CrawlAgent["CrawlAgent"]:::agent
        end
        ThinkingAgent["Observations/ThinkingAgent"]:::agent
        WriterAgent["WriterAgent"]:::agent
        LongWriterAgent["LongWriterAgent"]:::agent
        ProofreaderAgent["ProofreaderAgent"]:::agent
    end

    %% Tools Layer
    subgraph "Tools" 
        direction TB
        WebSearchTool["Web Search Tool"]:::tool
        CrawlWebsiteTool["Website Crawler Tool"]:::tool
    end

    %% Config & Utilities
    subgraph "Config & Utilities"
        direction TB
        LLMConfig["LLMConfig"]:::config
        Parsing["Parsing Utilities"]:::config
        MdToPDF["Markdown→PDF Utility"]:::config
        OSUtils["OS Utilities"]:::config
    end

    %% External Systems
    subgraph "External Systems"
        direction TB
        LLMProviders["LLM Providers"]:::external
        SearchProviders["Search Providers"]:::external
        Env[".env (API Keys)"]:::external
    end

    %% Connections
    User -->|uses CLI| CLI
    User -->|uses API| PythonAPI

    CLI -->|invokes| IterativeResearcher
    CLI -->|invokes| DeepResearcher
    PythonAPI -->|calls| IterativeResearcher
    PythonAPI -->|calls| DeepResearcher

    DeepResearcher -->|plan report| PlannerAgent
    PlannerAgent -->|outline| DeepResearcher
    DeepResearcher -->|spawn loops| IterativeResearcher

    IterativeResearcher -->|knowledge gaps| KnowledgeGapAgent
    KnowledgeGapAgent -->|gaps| IterativeResearcher
    IterativeResearcher -->|select tools| ToolSelectorAgent
    ToolSelectorAgent -->|tool list| IterativeResearcher

    IterativeResearcher -->|execute| SearchAgent
    IterativeResearcher -->|execute| CrawlAgent
    SearchAgent -->|fetch data| WebSearchTool
    CrawlAgent -->|fetch data| CrawlWebsiteTool
    WebSearchTool -->|results| SearchAgent
    CrawlWebsiteTool -->|results| CrawlAgent

    SearchAgent -->|findings| ThinkingAgent
    CrawlAgent -->|findings| ThinkingAgent
    ThinkingAgent -->|synthesized| KnowledgeGapAgent

    IterativeResearcher -->|draft section| WriterAgent
    IterativeResearcher -->|draft section| LongWriterAgent
    DeepResearcher -->|collect drafts| ProofreaderAgent
    ProofreaderAgent -->|final report| User

    %% Config flows
    Env -->|provides keys| LLMConfig
    LLMConfig -->|configures| IterativeResearcher
    LLMConfig -->|configures| DeepResearcher
    IterativeResearcher -->|uses| Parsing
    IterativeResearcher -->|uses| MdToPDF
    IterativeResearcher -->|uses| OSUtils

    %% Tool Extensions
    ToolSelectorAgent -->|loads| WebSearchTool
    ToolSelectorAgent -->|loads| CrawlWebsiteTool

    %% Parallel indication
    DeepResearcher -.->|parallel sections| IterativeResearcher

    %% Click Events
    click PythonAPI "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/iterative_research.py"
    click PythonAPI "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/deep_research.py"
    click CLI "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/main.py"
    click IterativeResearcher "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/iterative_research.py"
    click DeepResearcher "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/deep_research.py"
    click PlannerAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/planner_agent.py"
    click KnowledgeGapAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/knowledge_gap_agent.py"
    click ToolSelectorAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/tool_selector_agent.py"
    click SearchAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/tool_agents/search_agent.py"
    click CrawlAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/tool_agents/crawl_agent.py"
    click ThinkingAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/thinking_agent.py"
    click WriterAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/writer_agent.py"
    click LongWriterAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/long_writer_agent.py"
    click ProofreaderAgent "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/proofreader_agent.py"
    click LLMConfig "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/llm_config.py"
    click Parsing "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/agents/utils/parse_output.py"
    click MdToPDF "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/utils/md_to_pdf.py"
    click OSUtils "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/utils/os.py"
    click WebSearchTool "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/tools/web_search.py"
    click CrawlWebsiteTool "https://github.com/qx-labs/agents-deep-research/blob/main/deep_researcher/tools/crawl_website.py"

    %% Styles
    classDef frontend fill:#D6E9FE,stroke:#1C78C0,color:#1C78C0;
    classDef orchestrator fill:#FEF3D6,stroke:#D69F00,color:#D69F00;
    classDef agent fill:#E8F5E9,stroke:#2E7D32,color:#2E7D32;
    classDef tool fill:#F3E5F5,stroke:#6A1B9A,color:#6A1B9A;
    classDef config fill:#FFF3E0,stroke:#EF6C00,color:#EF6C00;
    classDef external fill:#ECEFF1,stroke:#455A64,color:#455A64;
```
