How stanford-oval/datastorm works
DataSTORM accepts a research question, explores it through a database-oriented search tree, consolidates findings, and generates a cited report; web research can add context. The README identifies the SQL agent and research pipeline as the two core components. The graph preserves those stages and the documented optional SUQL, sandbox, and upload capabilities. Some relationships are documented rather than visible in sampled implementation excerpts, so those edges cite the README; unsampled subsystem internals are represented only at supported directory paths.
Main entry points: agent.py SQL Agent · datatalk_basic_article_generation_costorm.py CLI / API · knowledge_curation.py Knowledge Curation · engine.py DataSTORM Engine
Components 19
Research Pipeline
- CLI / API
datatalk_basic_article_generation_costorm.py - DataSTORM Engine
knowledge_storm/datastorm/engine.py - Knowledge Curation
knowledge_storm/datastorm/modules/knowledge_curation.py - Tree Simulation
knowledge_storm/datastorm/modules/tree_simulator.py - Tree Search
knowledge_storm/datastorm/modules/tree_search_utils.py - Retriever
knowledge_storm/datastorm/modules/retriever.py - Research State
knowledge_storm/datastorm/modules/storm_dataclass.py - Co-STORM Engine
knowledge_storm/collaborative_storm/engine.py - Warm Start
knowledge_storm/collaborative_storm/modules/warmstart_hierarchical_chat.py
SQL Agent
- SQL Agent
knowledge_storm/datatalk_agent/agent.py - Agent Library
knowledge_storm/datatalk_agent/library.py - SQL State
knowledge_storm/datatalk_agent/state.py - Python Sandbox
knowledge_storm/datatalk_agent/python_modules/sandbox_runner.py
Report Generation
- Report Generation
final_report_gen_utils.py - Research Artifacts
datatalk_basic_article_generation_costorm.py
External Integrations
- Domain Database
- SUQL Services
- Web Search
knowledge_storm/rm.py
Other components
- Researcher
Connections 20
- Researcher → CLI / API: submits question
- CLI / API → DataSTORM Engine: starts research
- CLI / API → Co-STORM Engine: configures runner
datatalk_basic_article_generation_costorm.py - CLI / API → Knowledge Curation: uses curation
datatalk_basic_article_generation_costorm.py - CLI / API → Report Generation: uses generation
datatalk_basic_article_generation_costorm.py - DataSTORM Engine → Knowledge Curation: runs curation
knowledge_storm/datastorm/engine.py - DataSTORM Engine → Retriever: constructs retriever
knowledge_storm/datastorm/engine.py - Knowledge Curation → Tree Simulation: uses simulator
knowledge_storm/datastorm/modules/knowledge_curation.py - Tree Simulation → Tree Search: uses utilities
knowledge_storm/datastorm/modules/tree_simulator.py - Knowledge Curation → Research State: updates research state
knowledge_storm/datastorm/modules/knowledge_curation.py - Retriever → Web Search: queries sources
- Knowledge Curation → SQL Agent: asks database questions
- SQL Agent → SQL State: updates state
knowledge_storm/datatalk_agent/agent.py - Agent Library → SQL Agent: invokes agent
knowledge_storm/datatalk_agent/library.py - SQL Agent → Domain Database: executes SQL
- SQL Agent → SUQL Services: uses semantic queries
- SQL Agent → Python Sandbox: delegates analysis
knowledge_storm/datatalk_agent/agent.py - DataSTORM Engine → Report Generation: generates report
- Report Generation → Research Artifacts: writes artifacts
datatalk_basic_article_generation_costorm.py - Co-STORM Engine → Warm Start: runs warm start
knowledge_storm/collaborative_storm/engine.py