How qx-labs/agents-deep-research works
Project Type and Purpose
• This is an open-source Python library and CLI tool for “agentic deep research,” built on top of the OpenAI Agents SDK.
• It orchestrates multiple specialized agents (knowledge-gap, planner, tool-selector, tool-agents, writer, proofreader) to perform iterative or parallelized research and synthesize reports.
Top-Level Architecture
• User Interfaces:
• Python API (IterativeResearcher, DeepResearcher classes)
• CLI entry point (deep_researcher/main.py)
• Core Orchestrators:
• IterativeResearcher (single-topic loop)
• DeepResearcher (outline + parallel sub-research loops + final proofing)
• Agent Layer:
• PlannerAgent
• KnowledgeGapAgent
• ToolSelectorAgent
• ToolAgents (SearchAgent, CrawlAgent, etc.)
• Observations/ThinkingAgent
• WriterAgent
• ProofreaderAgent
• Tools Layer:
• Web search (deep_researcher/tools/web_search.py)
• Website crawler (deep_researcher/tools/crawl_website.py)
• Custom tools (extendable)
• Config & Utilities:
• LLMConfig (deep_researcher/llm_config.py)
• Parsing, markdown→PDF, OS utils
• External Systems:
• LLM providers (OpenAI, Azure, Anthropic, Gemini, etc.)
• Search providers (Serper, OpenAI tool calling)
• Optional .env for API keys
Component Relationships & Data Flows
• User input (CLI or API) → chosen Researcher (Iterative or Deep)
• DeepResearcher:
1. Calls PlannerAgent → generates report outline (sections)
2. Spawns multiple IterativeResearcher instances in parallel (one per section)
3. Each IterativeResearcher loops:
a. KnowledgeGapAgent analyzes current draft → gaps
b. ToolSelectorAgent chooses which ToolAgents to invoke
c. ToolAgents execute web searches or crawls → new findings
d. ObservationsAgent synthesizes findings → feeds back to KnowledgeGapAgent
e. After loop ends, WriterAgent drafts section
4. ProofreaderAgent combines section drafts → final report
• IterativeResearcher (stand-alone):
• Same inner loop (KnowledgeGap → Selector → Tools → Observations)
• Ends with WriterAgent output
Architectural Patterns & Design Principles
• Modular “agent” components: each agent encapsulates a single responsibility.
• Orchestration layer (IterativeResearcher / DeepResearcher) implements the control flow.
• Parallelism: DeepResearcher runs section-level researchers concurrently.
• Extension points: ToolAgents and custom tools can be dropped in without modifying core logic.
• Async I/O: Uses Python asyncio for parallel tool calls and model API requests.
• Configuration Abstraction: LLMConfig separates model-provider specifics from orchestration logic.
Technologies & Frameworks
• Python 3, asyncio
• OpenAI Agents SDK
• External LLM APIs (OpenAI, Azure, Anthropic, Gemini, etc.)
• Serper API (for Google search) or OpenAI tool calling
• Markdown & PDF utilities