How facebookresearch/docagent works
Project type and purpose
• DocAgent is a Python-based, full-stack system for automated, context-aware docstring generation. It provides both a CLI and a web UI for driving a multi-agent pipeline that reads a codebase, orchestrates a set of specialized agents (Reader, Searcher, Writer, Verifier), invokes external LLMs, and writes high-quality docstrings back into the code. It also includes a separate evaluation subsystem with its own web UI to assess docstring quality.
File structure
• Top-level “generate_docstrings.py” and “run_web_ui.py” launch the generation pipeline (CLI vs. web).
• “config/example_config.yaml” → user config for LLM endpoints, model names, API keys.
• “src/agent” → core agent framework: Orchestrator + four agent types + LLM factory and rate limiter.
• “src/dependency_analyzer” → builds file dependency DAG to traverse code hierarchically.
• “src/data/parse” → helpers for downloading repos, building AST, normalizing data.
• “src/evaluator” + “src/visualizer” → static analyzers (completeness, helpfulness, truthfulness) and progress/status display code.
• “src/web” → Flask app for generation UI (static assets + JS + HTML templates).
• “src/web_eval” → separate Flask app for evaluation UI.
README for additional insights
• Methodology: hierarchical traversal (dependency_analyzer) + agentic system (agent orchestrator + four agents).
Main components and their interactions
a. Configuration loader
• Reads “config/agent_config.yaml” → provides LLM endpoints, model names, generation settings.
b. Entry points
• CLI: generate_docstrings.py
• Web UI: run_web_ui.py → Flask app (src/web/app.py)
• Evaluation Web UI: src/web_eval/app.py
c. Core pipeline
• Orchestrator (src/agent/orchestrator.py)
– Calls dependency_analyzer to sort files
– For each file/module:
· Reader (loads code, builds AST context)
· Searcher (external/internal context retrieval)
· Writer (formulates prompts, calls LLM via LLMFactory)
· Verifier (validates and refines docstrings)
d. LLM abstraction
• LLMFactory routes to openai_llm, claude_llm, gemini_llm, huggingface_llm
• Rate limiter module throttles calls
e. Persistence & output
• Docstrings are injected back into source files
• CLI logs or Web UI progress shown by visualizer modules
f. Evaluation subsystem
• Evaluator modules compute completeness, helpfulness, truthfulness
• Web-based UI shows metrics and allows manual review
Architectural patterns & design principles
• Layered architecture:
• Presentation layer: CLI & two web UIs (Flask + static frontend)
• Application layer: Orchestrator and agents
• Service layer: LLM adapters, AST tools, dependency analyzer
• Data layer: file I/O, configuration, evaluation outputs
• Modular, single-responsibility agents
• Dependency-driven (hierarchical) traversal to maximize context accuracy
• Clear separation of concerns between generation vs. evaluation