How aiming-lab/autoresearchclaw works
1. Overall shape
- AutoResearchClaw is a Python 3.11+ research orchestration system that turns a topic into a paper through a staged pipeline.
- The core product is the researchclaw package; everything else supports the pipeline with adapters, sandboxes, domain profiles, UI, and benchmarking.
2. Primary control plane: the pipeline
- The main execution path lives under researchclaw/pipeline/.
- It is a 23-stage workflow grouped into scoping, literature discovery, synthesis, experiment design/execution, analysis, writing, and finalization.
- This is the central backbone for any architecture diagram: nearly all subsystems plug into or are called by this pipeline.
3. Pipeline inputs and configuration
- Configuration comes from YAML and env vars (config.researchclaw.example.yaml, prompts.default.yaml, domain profiles, credentials loaders).
- The configuration determines model provider, domain, experiment mode, export target, HITL policy, and optional bridges such as MetaClaw/OpenClaw.
- Domain-specific profiles under researchclaw/domains/profiles/ steer the pipeline for ML, biology, chemistry, physics, HEP, neuroscience, etc.
4. Research intelligence layer
- Literature and web discovery are separate from experiment execution.
- researchclaw/literature/ handles arXiv, OpenAlex, Semantic Scholar, novelty, trends, verification, and caching.
- researchclaw/web/ provides crawler/search/PDF extraction/scholar access, and researchclaw/knowledge/ stores extracted entities, relations, and queryable research memory.
5. Experiment generation and execution
- Experiment work is split across researchclaw/experiment/ and researchclaw/agents/.
- code_searcher, benchmark_agent, figure_agent, and benchmark_agent create runnable code, select datasets/baselines, and generate publication figures.
- The pipeline can run experiments in multiple backends: local sandbox, Docker, SSH remote, Colab, or specialized domain sandboxes.
6. Domain execution boundaries
- Domain adapters in researchclaw/domains/adapters/ decide how the pipeline executes for a field.
- Most domains use generic/Python-based execution, but some have specialized paths:
- biology uses COBRApy-style metabolic workflows,
- chemistry can use RDKit,
- HEP routes through ColliderAgent/Magnus-style physics tooling,
- statistics uses simulation-study workflows.
- This domain boundary matters because it changes the runtime, toolchain, and artifact shape of experiment stages.
7. Human-in-the-loop and copilot flow
- researchclaw/hitl/ is a substantial subsystem that can pause, branch, approve, reject, guide, or collaborate on the pipeline.
- It is exposed through CLI, WebSocket, and MCP adapters, and interacts with stage gates and SmartPause/cost guardrails.
- Diagram-wise, this is an overlay on the main pipeline: it intercepts selected stages rather than replacing the pipeline itself.
8. LLM and agent backends
- researchclaw/llm/ abstracts model access, including openai-compatible providers and ACP agent backends.
- The system can also delegate to external CLI agents via ACP/OpenClaw-compatible paths.
- Prompting is domain-aware via researchclaw/prompts/, and skills from researchclaw/skills/ plus .claude/skills/ are injected to guide the model behavior.
9. Quality, verification, and anti-fabrication
- Several subsystems act as guards rather than generators:
- researchclaw/assessor/ scores papers and recommends venues,
- researchclaw/experiment/validator.py and verified_registry.py enforce evidence-backed outputs,
- researchclaw/pipeline/experiment_diagnosis.py and experiment_repair.py detect and repair failed experiments,
- citation verification and claim checking prevent fabricated references or numbers.
- These are important boundaries: generated content is not trusted until it passes validation.
10. Persistence and memory
- State is persisted in multiple layers:
- run artifacts in artifacts/,
- knowledge base entries in researchclaw/knowledge/,
- long-term memory in researchclaw/memory/,
- collaboration/cross-run sharing via researchclaw/collaboration/ and optional MetaClaw bridge.
- This creates a feedback loop where prior runs influence future runs through lessons, skills, and cached retrieval.
11. Output/export layer
- Final deliverables are produced by researchclaw/templates/ and researchclaw/report.py.
- Outputs include Markdown paper drafts, BibTeX, LaTeX conference papers, charts, reviews, verification reports, and archived experiment results.
- researchclaw/overleaf/ adds sync/formatting support for LaTeX workflows, so the export boundary is a true handoff to publication tooling.
12. Orchestration surfaces
- The repository exposes several entry points:
- CLI (researchclaw/cli.py, __main__.py),
- HTTP/server components in researchclaw/server/,
- websocket broadcaster/manager for live monitoring,
- researchclaw/dashboard/ for metrics and progress,
- frontend-legacy/ as an older UI shell,
- website/ for public docs/marketing.
- For architecture diagrams, these are external interaction surfaces around the same core pipeline, not separate engines.
13. Benchmarking and reproducibility
- experiments/arc_bench/ is an embedded benchmark suite with manifests, rubrics, adapters, and run/evaluation scripts.
- It exercises the same core pipeline against controlled topics, making the repo both a product and an evaluation platform.
- tests/ is extensive and covers nearly every subsystem, reinforcing that the architecture is modular and contract-driven.
14. Infrastructure dependencies
- The system depends materially on Docker, optional SSH/SLURM/cloud executors, LaTeX toolchains, and external scholarly APIs.
- OpenClaw, ACP-compatible CLIs, and optional messaging/web bridges extend the system beyond a single-process tool into a distributed, human-agent collaborative platform.
Main entry points: stage_impls/__init__.py Stage impls · runner.py Pipeline runner · hitl/__init__.py HITL · report.py Report
Components 32
Core pipeline
Primary research-to-paper orchestration backbone.
- Pipeline runner · Main workflow entry and stage sequencing for research runs.
researchclaw/pipeline/runner.py - Stages · Defines the staged research lifecycle and stage boundaries.
researchclaw/pipeline/stages.py - Stage impls · Implementation modules for topic, literature, synthesis, execution, analysis, writing, and review stages.
researchclaw/pipeline/stage_impls/__init__.py - Contracts · Shared data contracts and interfaces used across pipeline stages.
researchclaw/pipeline/contracts.py
Research intelligence
Literature, web, knowledge, and trend subsystems that feed topic selection and synthesis.
- Literature · ArXiv, OpenAlex, Semantic Scholar, novelty, trends, and verification.
researchclaw/literature/__init__.py - Web · Crawler, search, PDF extraction, and scholar access for web-backed discovery.
researchclaw/web/__init__.py - Knowledge graph · Entities, relations, query, and visualized research memory.
researchclaw/knowledge/__init__.py - Trends · Feeds, opportunity finding, and daily digests for topic selection.
researchclaw/trends/__init__.py
Execution and agents
Experiment backends, agent builders, and domain-specific runtime boundaries.
- Experiment runtime · Sandbox factories, runners, validators, and backend execution environments.
researchclaw/experiment/__init__.py - Domain adapters · Field-specific execution paths for biology, chemistry, physics, ML, statistics, and related domains.
researchclaw/domains/adapters/__init__.py - LLM layer · Provider abstraction for model calls, ACP compatibility, and agent backends.
researchclaw/llm/__init__.py - Prompts · Domain-aware prompt selection and assembly.
researchclaw/prompts/__init__.py - Agents · Code search, benchmark selection, and figure generation agents used by the pipeline.
researchclaw/agents/__init__.py
Validation and HITL
Human-in-the-loop controls, claim checks, and repair/verification guardrails.
- Verified registry · Tracks evidence-backed artifacts and trusted outputs.
researchclaw/pipeline/verified_registry.py - Repair · Diagnoses broken experiments and attempts automated repair.
researchclaw/pipeline/experiment_repair.py - Diagnosis · Detects experiment failures and classifies remediation needs.
researchclaw/pipeline/experiment_diagnosis.py - Paper verifier · Checks paper outputs, citations, and evidence consistency.
researchclaw/pipeline/paper_verifier.py - HITL · Interactive approval, branching, pausing, and collaboration overlay on pipeline stages.
researchclaw/hitl/__init__.py - HITL adapters · CLI, WebSocket, MCP, and scripted interfaces for human intervention.
researchclaw/hitl/adapters/__init__.py
Surfaces and services
CLI, server, dashboard, voice, and copilot entry points.
- CLI · Command-line interface for running the system.
researchclaw/cli.py - Server · HTTP routes, chat, pipeline, projects, voice, and websocket management.
researchclaw/server/app.py - Dashboard · Metrics, collection, and progress broadcasting for live runs.
researchclaw/dashboard/__init__.py - Copilot · Branching, feedback, and mode control for guided operation.
researchclaw/copilot/__init__.py - Voice · Voice command and transcription interface for interaction.
researchclaw/voice/__init__.py
Output and memory
Reporting, templates, long-term memory, and collaboration persistence.
- Collaboration · Cross-run sharing, publish/subscribe, and deduplication support.
researchclaw/collaboration/__init__.py - Memory · Embedding-backed memory, retrieval, and decay for prior runs and writing.
researchclaw/memory/__init__.py - Report · Final reporting and artifact assembly for paper outputs.
researchclaw/report.py - Templates · Conference and results-table templates for publication handoff.
researchclaw/templates/__init__.py - Overleaf · Formatting, sync, and conflict handling for LaTeX workflows.
researchclaw/overleaf/__init__.py
Benchmarks and external
Embedded benchmark harnesses, skills, and external agent bridges.
- ARC bench · Embedded benchmark harness for comparative evaluation and reproducibility.
experiments/arc_bench/README.md - Builtin skills · Packaged domain, experiment, and tooling skills injected into agents and prompts.
researchclaw/skills/registry.py - External agents · Optional bridges to domain agent ecosystems such as biology, collider, and statistical research agents.
external/agents/README.md
Connections 33
- Pipeline runner → Stages: drives · Runs the staged research workflow.
- Pipeline runner → Stage impls: dispatches · Calls stage-specific implementations.
- Pipeline runner → Contracts: uses · Shares typed data contracts across stages.
- Stage impls → Literature: queries · Discovery and novelty stages pull literature intelligence.
- Stage impls → Web: searches · Web-backed retrieval and PDF extraction support discovery.
- Stage impls → Knowledge graph: writes · Extracts entities and relations into research memory.
- Stage impls → Trends: selects topics · Topic and opportunity stages use trend signals.
- Stage impls → Experiment runtime: executes · Experiment design and execution stages launch sandboxes and runners.
- Experiment runtime → Domain adapters: routes through · Domain adapters choose runtime and toolchain boundaries.
- Stage impls → Agents: invokes · Code search, benchmarks, and figure generation support experiments and writing.
- Agents → LLM layer: calls · Agents use model backends for reasoning and generation.
- Stage impls → Prompts: formats · Stages assemble domain-aware prompts.
- HITL → Pipeline runner: intercepts · Human approvals and pauses overlay selected pipeline stages.
- HITL → HITL adapters: exposes · Interaction surfaces for CLI, WS, MCP, and scripted control.
- HITL → Verified registry: guards · Checks evidence and trust before advancing.
- Repair → Diagnosis: feeds · Diagnosis drives repair decisions.
- Paper verifier → Verified registry: records · Only validated outputs become trusted artifacts.
- Stage impls → Paper verifier: verifies · Writing and finalization stages are checked before export.
- Memory → Pipeline runner: informs · Prior runs influence future orchestration and writing.
- Collaboration → HITL: shares · Cross-run collaboration and feedback flow into HITL.
- Stage impls → Report: exports · Finalization hands artifacts to reporting.
- Report → Templates: renders · Uses publication templates for paper output.
- Report → Overleaf: syncs · Supports LaTeX handoff and Overleaf workflows.
- CLI → Pipeline runner: starts · CLI is the primary local entry point.
- Server → Pipeline runner: hosts · HTTP API and websocket services wrap pipeline execution.
- Dashboard → Server: subscribes · Monitoring data is pushed through live services.
- Copilot → Pipeline runner: guides · Copilot provides branching and guided operation.
- Voice → Server: fronts · Voice is exposed through server-side interaction.
- ARC bench → Pipeline runner: evaluates · Benchmark runs exercise the same core pipeline.
- Builtin skills → Prompts: injects · Skills feed prompting and agent behavior.
- Builtin skills → Agents: guides · Packaged skills shape agent specialization.
- External agents → Domain adapters: extends · Optional external agent ecosystems augment domain execution.
- External agents → LLM layer: connects · External agents may be reached through model-compatible paths.