How vonhatthinh/deep-research-agent works
Project Type and Purpose
• This is a full-stack Python application consisting of a FastAPI backend and a Streamlit frontend.
• It implements a multimodal “Deep Research Agent” pipeline: users submit text, images, or documents via the UI; the backend orchestrates a series of specialized agents (QueryAnalyzer, TaskPlanner, Researcher, Visualizer, Evaluator); results are streamed back in real time and assembled into a final report.
File Structure Overview
• Top-level:
• main.py – entry point for the FastAPI server
• ui/app.py – Streamlit frontend
• agent/ – core multi-agent modules (base_agent.py, multi_agent.py, schema.py, tools.py, agent.py)
• core/config.py – configuration handling
• database/vector_store.py – wrapper around a vector database for knowledge storage/retrieval
• resources/ – static assets (e.g., workflow_diagram.png)
• .devcontainer, DockerFile – dev environment and containerization
• Patterns & Layers:
• Presentation Layer: Streamlit UI
• API Layer: FastAPI endpoints & SSE streaming
• Orchestration Layer: multi_agent.py coordinating agent pipeline
• Agent Modules: each agent implements one research stage
• Data Layer: vector_store + file‐upload handling
• External Integrations: OpenAI Assistants API, Tavily Search API, Gemini Vision, DALL-E3
Key Architectural Patterns
• Modular, single‐responsibility agents in a pipeline (QueryAnalyzer → TaskPlanner → Researcher → Visualizer → Evaluator)
• Asynchronous, streaming communication (FastAPI’s StreamingResponse + Streamlit’s st.write_stream)
• Clear separation of concerns: UI ↔ API ↔ Agents ↔ Data layer ↔ External services
• Pluggable “tools” abstraction for search, image generation, document parsing
Main Components & Their Relationships
A. Streamlit UI (ui/app.py)
• Submits user queries/files via HTTP POST to backend /query endpoint
• Listens for Server-Sent Events and renders the agent’s step‐by‐step output
B. FastAPI Backend (main.py + agent modules)
• Exposes /status and POST /query
• Parses inputs, persists/uploaded files or images if needed
• Invokes the MultiAgent orchestrator asynchronously
• Streams intermediate events back to UI
C. MultiAgent Orchestrator (agent/multi_agent.py)
• Imports specialized agents from agent/base_agent.py and agent/agent.py
• Uses agent/tools.py for external calls
• Emits events conforming to agent/schema.py
D. Vector Store (database/vector_store.py)
• Persists research snippets into a vector‐searchable index
• Queried by the Researcher agent for context or similarity retrieval
E. External Services
• OpenAI Assistants API for generative steps
• Tavily Search API for live web search
• Gemini Vision for image comprehension
• DALL-E 3 for image generation
Essential Details to Capture
• Entry points: ui/app.py and main.py.
• Agent pipeline sequence and the dual role of agent/tools.py (adapters to external APIs).
• Vector store interactions (read/write).
• File and image upload handling within the API layer.
• Asynchronous streaming flow.
• Environment setup (DockerFile, devcontainer) as a side note or deployment layer.