How psykick-21/deep-research works
Project Type and Purpose
• Deep Research is a Python-based research automation tool. It leverages LangChain/LangGraph to orchestrate multiple AI agents (nodes) that collaboratively gather, reflect on, and synthesize information into a structured research report. Users can interact via a CLI script (main.py) or a Jupyter notebook (main.ipynb).
Top-Level File Structure Analysis
• deep_research/
• configuration.py – loads settings and API keys
• graph.py – defines the LangGraph workflow (DAG orchestrator)
• nodes.py – implements individual research agents (planning, search, reflection, synthesis)
• prompts.py – houses the text prompts for LLM interactions
• state.py – manages in-memory or persisted research state
• struct.py – defines data models (sections, queries, results)
• utils.py – helper functions (logging, API wrappers)
• main.py / main.ipynb – entry points for batch or interactive runs
• temp/ – scratch notebook
• pyproject.toml – build & dependency config
Key Insights from README
• Two entry points: script and notebook
• Prereqs: Python 3.12, Poetry, four external APIs (OpenAI, Anthropic, Google AI, Tavily)
• Workflow: Initial planning → Section-wise research (loop of search, reflection, synthesis) → Final curation
• Outputs: logs/ (detailed process logs), reports/ (final research reports)
Main Components
a. User Interface
• CLI (main.py)
• Notebook (main.ipynb)
b. Configuration Loader
• Reads .env, configuration.py
c. Workflow Engine
• LangGraph orchestrator (graph.py)
d. Research Agents (Nodes)
• Structure Generator Agent
• Search Agent (Tavily + LLM knowledge base)
• Reflection Agent
• Query Generator Agent
• Synthesis Agent
e. State & Data Store
• In-memory state manager (state.py)
• Data models (struct.py)
f. External Services
• OpenAI, Anthropic, Google AI (LLM endpoints)
• Tavily (deep web search)
g. Utilities
• Logging subsystem (utils.py → logs/)
• Report writer (outputs to reports/)
Relationships and Data Flows
• User → starts main.py/notebook
• Configuration Loader → initializes API clients & graph parameters
• Workflow Engine → invokes Structure Generator → produces outline
• For each outline section:
• Search Agent ↔ External APIs → collects raw data
• Reflection Agent → reviews accumulated data
• Query Generator Agent → produces refined search queries
• Synthesis Agent → generates section text
• State Manager ← stores intermediate results
• Final Curation Step (LangGraph node) → merges all sections
• Report Writer → persists final document → logs progress
Architectural Patterns & Principles
• Agent-based pipeline: each node encapsulates a responsibility
• Directed acyclic workflow (LangGraph) for clear orchestration
• Recursive improvement loop for each section
• Separation of concerns: config, orchestration, node logic, state management, I/O