How assafelovic/gpt-researcher works
Overview
GPT Researcher is an AI research application and Python package for producing citation-backed reports from web sources or local documents. Its documented model is planner/execution agents: plan questions, gather sources, summarize and track citations, then aggregate a report. Research can include parallel work, image generation, deep recursive exploration, and multiple report formats.
Public Entry Points
• gpt_researcher/__init__.py: package-facing API; the README documents from gpt_researcher import GPTResearcher, followed by conduct_research() and write_report().
• main.py: documented FastAPI/ Uvicorn application entry point (uvicorn main:app), serving the backend-facing web experience.
• cli.py: repository-level command-line entry point area; the tree establishes its existence, but supplied evidence does not define its commands.
• frontend/nextjs: production-oriented Next.js application with API routes and research views.
• multi_agents/main.py: alternative multi-agent workflow entry point, distinct from the primary package flow.
Core Components
• Research agent and workflow orchestration — gpt_researcher/agent.py
Owns the central research-agent surface used by package callers. It coordinates the research task and report-writing capabilities described in the README.
• Planning, execution, and report generation — gpt_researcher/actions
Contains task/query processing, agent creation, retrieval coordination, web scraping actions, markdown processing, and report generation areas. These correspond to the documented planner, execution, summarization, and publisher responsibilities.
• Search and source adapters — gpt_researcher/retrievers
Organizes adapters for numerous search providers and specialized sources, including Tavily, Bing, Google, academic indexes, custom retrievers, and MCP-backed retrieval. These are integration points rather than one mandatory provider.
• Web acquisition and document extraction — gpt_researcher/scraper
Contains browser- and HTTP-oriented scrapers plus handlers for PDFs, JavaScript pages, ArXiv, Firecrawl, and other extraction strategies.
• Context, documents, and knowledge state — gpt_researcher/context
Owns context retrieval/compression concerns. Related document loaders and vector-store concerns are organized under gpt_researcher/document and gpt_researcher/vector_store; the README states that local documents and memory/context are supported. The supplied evidence does not establish a durable database.
• Models and runtime configuration — gpt_researcher/llm_provider
Provides model-provider areas, including generic LLM and image-generation support. Configuration and environment-backed settings are organized under gpt_researcher/config.
• HTTP, reporting, and realtime server — backend/server
Contains the application server, agent discovery, report storage, multi-agent running, and WebSocket management areas. It is the backend boundary for the documented local web deployment.
• Web client — frontend/nextjs
Provides the feature-rich Next.js UI, research pages, settings, report/chat views, and API route surface. The README also documents a separate lightweight frontend served by the backend.
Relationships
A caller supplies a query, configuration, optional local-document source, and provider credentials to the research agent. The orchestration layer plans work and delegates source acquisition to retrievers and scrapers; context/document components provide collected material for model processing. The resulting report and citations are returned to the caller or exposed through the backend, with report storage and WebSocket updates supporting the web client.
Optional and Alternative Boundaries
gpt_researcher/mcp adds caller-configured MCP data sources alongside web retrieval. Image generation is an optional model capability. deep_agents provides a separate recursive deep-research implementation, while multi_agents contains LangGraph/AG2-style specialized-agent workflows; neither should be assumed to be the default package path.
Main entry points: actions Actions · agent.py Research agent · context Context · server Backend server
Components 15
Research package
Core planning, acquisition, processing, and model capabilities.
- Package API · Exposes GPTResearcher to package callers.
gpt_researcher/__init__.py - Research agent · Coordinates research and report writing.
gpt_researcher/agent.py - Actions · Plans tasks, retrieves material, summarizes results, and generates reports.
gpt_researcher/actions - Retrievers · Provides search-provider and specialized-source integrations.
gpt_researcher/retrievers - Scraper · Extracts content from web pages and documents.
gpt_researcher/scraper - Context · Retrieves and compresses research context.
gpt_researcher/context - Documents · Supports local-document research inputs.
gpt_researcher/document - Vector store · Supports vector-store concerns for context and documents.
gpt_researcher/vector_store - LLM provider · Provides language-model and optional image-generation capabilities.
gpt_researcher/llm_provider - Configuration · Organizes environment-backed settings.
gpt_researcher/config - MCP sources · Adds caller-configured MCP data sources.
gpt_researcher/mcp
Web deployment
Backend and frontend surfaces for web-based research.
- Backend server · Provides application serving, report storage, agent running, and WebSocket management.
backend/server - Server entry point · Runs the documented Uvicorn application.
main.py - Web client · Provides research views, settings, reports, chat, and API routes.
frontend/nextjs
Other components
- Caller · Supplies queries, configuration, credentials, and optional local documents.
Connections 14
- Caller → Package API: supplies · The caller supplies the research request and inputs.
- Package API → Research agent: exposes · The package API exposes GPTResearcher.
- Research agent → Actions: orchestrates · The research agent coordinates planning and report-writing actions.
- Actions → Retrievers: delegates · Actions delegate source acquisition to retrievers.
- Actions → Scraper: delegates · Actions delegate web and document extraction to scrapers.
- Context → Actions: provides context · Context components provide collected material for model processing.
- Documents → Context: supplies documents · Document loaders support local-document context.
- Vector store → Context: supports · Vector-store concerns support context retrieval.
- LLM provider → Actions: supports processing · Model providers support summarization and report generation.
- Configuration → Research agent: configures · Runtime settings configure the research workflow.
- MCP sources → Retrievers: extends · MCP adds caller-configured sources alongside web retrieval.
- Server entry point → Backend server: starts · The FastAPI entry point starts the backend application.
- Research agent → Backend server: exposes reports through · Research results and citations are exposed through the backend.
- Web client → Backend server: uses · The web client uses backend APIs and realtime services.