How alpha-innovator/dolphin works
Project Type and Purpose
• Dolphin is a Python-based closed-loop, LLM-driven auto-research framework.
• It orchestrates iterative rounds of idea generation, code implementation, experiment execution, and results analysis, feeding feedback into subsequent rounds.
a. Orchestrator
• launch_dolphin.py / launch_dolphin.sh
• Coordinates the closed-loop workflow and round management.
b. Idea Generation Module
• dolphin_utils/generate_ideas.py
• Uses prompts (dolphin_utils/prompts.py) and LLM utilities (dolphin_utils/llm_utils.py) to produce novel research ideas.
c. Retrieval & Context Module
• dolphin_utils/rag_tools/ (lit_review.py, lit_review_tools.py, utils.py, prompts.py)
• Retrieves and ranks relevant papers; provides contextual feedback to the LLM.
d. Code Template & Implementation Module
• dolphin_utils/experiments_utils.py, dolphin_utils/rag_utils.py
• Builds/refines code templates for running experiments; handles exception-traceback guided debugging.
e. Experiment Runner
• examples/… directories (image_classification_cifar100, point_classification_modelnet, sentiment_classification_sst2)
• Scripts (experiment.py, launcher.sh) that train/evaluate models and produce result artifacts (best.pth, final_info.json).
f. Results Analyzer
• Parses final_info.json and other metrics (e.g., metrics.py); summarizes performance improvements.
g. Data & Artifacts Store
• experiment directories under examples/, run_0/, seed_ideas.json
• Stores generated code, logs, model checkpoints, and analysis outputs.
h. External Services
• LLM API (OpenAI or self-hosted via Ollama per docs/ollama_doc.md)
• Datasets (downloaded by users, not bundled)
Relationships and Data Flow
• Orchestrator → Idea Generation: “start round, pass previous round feedback”
• Idea Generation → Retrieval Module: “fetch literature context”
• Idea Generation → Code Template Module: “produce initial code skeleton”
• Code Template Module → Experiment Runner: “execute experiment script”
• Experiment Runner → Data Store: “write checkpoints, final_info.json”
• Experiment Runner → Results Analyzer: “send raw metrics”
• Results Analyzer → Orchestrator: “send summary feedback for next idea generation”
• Orchestrator loops back to Idea Generation
Architectural Patterns and Principles
• Closed-loop feedback architecture: clearly illustrate the iterative cycle.
• Modular separation: LLM interaction, retrieval, code synthesis, execution, analysis are separate modules.
• Pluggable backends: experiments defined per task under examples/; supports adding new tasks.
• Declarative configuration: launch scripts and prompt JSONs define metadata.
Relevant Technologies & Frameworks
• Python 3.11, Conda environment
• Hugging Face/LLM APIs or Ollama self-hosted model
• PyTorch (implied by .pth model files)
• RAG tools for retrieval-augmented generation
• Shell scripts for orchestration (bash)
4. To the left of Retrieval, place Idea Generation (rectangle), connected from Orchestrator and Retrieval.
5. Below Idea Generation, place Code Template & Implementation (rectangle).