How sakanaai/ai-scientist-v2 works
Main components
• Core Research Pipeline – This is the overall “brain” of the system that generates hypotheses, runs experiments, reviews results, and produces manuscripts.
• Tree Search Module – Represented by the “treesearch” directory, this includes sub-components such as:
• Agent Manager (agent_manager.py)
• Backend connectors for interfacing with LLM APIs (backend/backend_openai.py, backend/backend_anthropic.py)
• Utilities for configuration, serialization, and visualization (in treesearch/utils and viz_templates)
• Supporting scripts for journals, interpreters, and debugging (journal.py, interpreter.py, log_summarization.py)
• LLM and VLM Interaction – Files like llm.py and vlm.py indicate modules managing language models and vision-language models, plus associated processing scripts (e.g., perform_llm_review.py, perform_vlm_review.py, perform_writeup.py).
• Idea and Experiment Management – The “ideas” directory holds JSON and Python files for research ideas. Additionally, “fewshot_examples” exemplify input/output formats for experiments.
• Template/Manuscript Output – The “blank_icbinb_latex” and “blank_icml_latex” directories provide LaTeX templates for generated papers.
• External API Tools – The “tools” directory, including modules such as semantic_scholar.py, deals with external services for citation and literature search.
Out the relationships and interactions
• The launch entry point (launch_scientist_bfts.py) ties all operations together.
• The agentic tree search (the treesearch module) orchestrates the decision process: using a multi-worker (agent_manager.py and parallel_agent.py) search that explores hypothesis nodes based on configuration in bfts_config.yaml.
• There is a clear use of backend modules to abstract calls to external LLM APIs (OpenAI and Anthropic via AWS Bedrock), with environment variables (e.g., OPENAI_API_KEY, AWS keys) configuring these interactions.
• The experimental phase interacts with the idea files from the “ideas” directory as input and produces visualization outputs and logs (e.g., unified_tree_viz.html within experiment logs).
• Tools such as the Semantic Scholar connector often provide feedback to check novelty and retrieve citations, linking the research process with relevant external literature.
• Utility scripts (e.g., perform_plotting.py, perform_writeup.py) are involved for different stages of processing (analysis, writing, plotting).
Important design principles
• Modularity – Each functionality (tree search, LLM interactions, template management) is separated into its own module.
• Agentic Orchestration – The tree search with multiple agents denotes an asynchronous, parallel decision-making architecture.
• Extensibility – Presence of directories like “ideas” and “fewshot_examples”, and clear configuration in YAML, suggests that research topics and experiment parameters are easily modifiable.
• Safety and Flexibility – Given warnings in the README, the system is designed to execute code generated by LLMs, so risk boundaries and environment isolation are critical, though these may be managed externally (e.g., via containerization or secure execution contexts).
Relevant technologies and frameworks
• Python (version 3.11 as recommended) with supporting libraries like PyTorch for GPU usage.
• CUDA for GPU processing on NVIDIA GPUs.
• LaTeX templates for manuscript generation.
• External API integrations (OpenAI, Amazon Bedrock for Claude, and Semantic Scholar).
• Configuration via YAML (bfts_config.yaml), which controls the tree search parameters.