# ruc-nlpir/arbor architecture

> Architecture diagram of the GitHub repository ruc-nlpir/arbor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ruc-nlpir/arbor
- Repository: https://github.com/ruc-nlpir/arbor

## Overview

1. **What this repo is**
   Arbor is a Python-first autonomous research system packaged as the `arbor` CLI. Its architecture is centered on a **Coordinator + Executor** split: the Coordinator plans and branches ideas, while Executors run isolated experiments and feed results back into the Idea Tree.

2. **Core runtime boundary**
   The main runtime lives under `src/core/` and provides shared infrastructure: agent abstractions, config loading/resolution, context management, LLM provider adapters, tool abstractions, logging, OAuth, and skill loading. This is the shared foundation used by both the Coordinator and Executor processes.

3. **Coordinator subsystem**
   `src/coordinator/` is the decision-making engine. It owns the **Idea Tree**, orchestrates the “arbor cycle” (observe → ideate → select → dispatch → backpropagate → decide), and applies higher-level policies like convergence, contamination checks, context pruning, and human-in-the-loop gating.

4. **Executor subsystem**
   `src/executor/` is the implementation/evaluation worker. It takes one hypothesis, edits code, runs the benchmark or task, and reports evidence. The important architectural boundary is that Executors operate in **isolated git worktrees**, so experimental changes stay separated from the main branch until a merge decision is made.

5. **Experiment and repository safety**
   The repo treats the target project as a sandboxed external workspace. Tools under `src/coordinator/tools/`, `src/core/tools/`, and `src/zoo/` enforce protected file access, git/worktree operations, evaluation execution, and integrity checks. This is the mechanism that keeps Arbor from directly mutating the user’s main repository state during exploration.

6. **CLI and user-facing control plane**
   `src/cli/` is the operator-facing shell. It handles startup, intake conversation, configuration, setup, login, doctoring, replay, export, run control, dashboard launching, and resume/picker flows. It is the main entry point for interactive use and the place where goals are turned into a concrete research contract.

7. **Session state, events, and observability**
   Arbor is event-driven. `src/events/` defines the event bus, payloads, subscribers, and stats collection. Runs emit structured events to session artifacts such as `events.jsonl`, tree snapshots, stats, and reports. Those artifacts are then consumed by replay, export, dashboard, and the web UI.

8. **Reporting and replay**
   `src/report/`, `src/dashboard.py`, `src/cli/replay.py`, and `src/cli/tree_export.py` turn session data into human-readable outputs. The architecture clearly separates **execution** from **presentation**: the run produces structured artifacts first, and the report/dashboard layers render them afterward.

9. **Search and novelty checking**
   Arbor can ground ideas in prior art through `src/core/tools/web/` and the search-oriented parts of `src/coordinator/tools/` and `src/search_agent/`. The built-in alphaXiv-backed path is used for novelty checks and related-work summaries, either manually via `arbor idea-check` or automatically when new ideas are added to the tree.

10. **Extension model: plugins, skills, and MCP**
   Domain customization comes from two layers: YAML **plugins** under `src/plugins/` and markdown **skills** under `skills/` and `src/skills/`. For keyless usage inside Claude Code/Codex-style hosts, `src/mcp/` exposes Arbor’s deterministic tools over MCP so the host model can drive the loop without Arbor needing its own API key.

11. **Benchmark zoo and task packaging**
   The benchmark ecosystem lives in `src/zoo/`, `arbor-zoo/`, and `examples/`. This layer handles acquiring benchmark assets, scaffolding new tasks, packing/verifying benchmark bundles, and defining task-specific eval contracts. It is the boundary where Arbor generalizes from “one repo optimization run” to reusable benchmark definitions.

12. **External service and model boundary**
   Model access is abstracted behind provider adapters in `src/core/llm/` and OAuth helpers in `src/core/oauth/`. The runtime can speak to Anthropic, OpenAI/Responses, OpenAI-compatible endpoints via LiteLLM-style backends, and keyless host-driven modes. This is a clean separation between Arbor’s orchestration logic and the actual model backend.

13. **Delivery surfaces**
   The repository also ships a static marketing/docs surface: MkDocs content under `docs/` and a React/Vite project under `project_page/` for the public website and demo assets. These are presentation-only surfaces; they don’t participate in the core research loop, but they mirror the same concepts in a browser-friendly form.

14. **How data flows end to end**
   A typical run flows from CLI intake → config resolution → Coordinator planning → Executor worktree execution → event emission and artifact capture → merge/prune decision → report/dashboard replay. The key architectural idea is that Arbor learns by accumulating structured tree state and run artifacts, not by keeping everything in transient chat context.

Main entry points: [`src/cli/app.py`](https://github.com/ruc-nlpir/arbor/blob/main/src/cli/app.py) (arbor CLI), [`src/coordinator/orchestrator.py`](https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/orchestrator.py) (Coordinator), [`src/executor/main.py`](https://github.com/ruc-nlpir/arbor/blob/main/src/executor/main.py) (Executor), [`src/events/bus.py`](https://github.com/ruc-nlpir/arbor/blob/main/src/events/bus.py) (Event bus)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["CLI"]
  node_src_cli_app["arbor CLI<br/>entrypoint<br/>[app.py]"]
  node_src_cli_commands["Commands<br/>command layer"]
  node_src_cli_intake["Intake flow<br/>interactive setup"]
  node_src_cli_resume["Resume tools<br/>session control<br/>[resume_picker.py]"]
end

subgraph group_runtime["Core runtime"]
  node_src_core_config["Config system<br/>configuration<br/>[config_resolve.py]"]
  node_src_core_agent["Agent core<br/>agent runtime<br/>[agent.py]"]
  node_src_core_tools["Tooling<br/>toolkit"]
  node_src_core_llm["LLM adapters<br/>model boundary"]
end

subgraph group_coord["Coordinator"]
  node_src_coordinator_main["Coordinator<br/>decision engine<br/>[orchestrator.py]"]
  node_src_coordinator_tree["Idea tree<br/>state model<br/>[idea_tree.py]"]
  node_src_coordinator_policies{{"Policies<br/>safety policy<br/>[contamination.py]"}}
  node_src_coordinator_tools["Coord tools"]
end

subgraph group_exec["Executor"]
  node_src_executor_main["Executor<br/>worker runtime<br/>[main.py]"]
end

subgraph group_obs["Events and reporting"]
  node_src_events_bus["Event bus<br/>event system<br/>[bus.py]"]
  node_src_report["Reports<br/>rendering<br/>[generator.py]"]
  node_src_dashboard["Dashboard<br/>ui server<br/>[dashboard.py]"]
  node_src_webui["Web UI<br/>frontend server<br/>[server.py]"]
end

subgraph group_ext["Extensions"]
  node_src_search_agent["Search agent<br/>novelty search<br/>[agent.py]"]
  node_src_plugins["Plugins<br/>yaml extensions"]
  node_skills_bundle["Skills<br/>prompt assets"]
  node_src_mcp["MCP server<br/>integration boundary<br/>[server.py]"]
end

subgraph group_bench["Benchmark zoo"]
  node_src_zoo["Benchmark zoo<br/>benchmark packager"]
  node_arbor_zoo["Zoo assets<br/>benchmark library"]
end

subgraph group_sites["Public surfaces"]
  node_docs["Docs<br/>documentation"]
  node_project_page["Website<br/>marketing site"]
end

node_src_cli_app -->|"dispatches"| node_src_cli_commands
node_src_cli_app -->|"starts intake"| node_src_cli_intake
node_src_cli_app -->|"resolves config"| node_src_core_config
node_src_cli_app -->|"launches loop"| node_src_coordinator_main
node_src_cli_app -->|"spawns workers"| node_src_executor_main
node_src_cli_app -->|"opens dashboard"| node_src_dashboard
node_src_cli_app -->|"serves tools"| node_src_mcp
node_src_coordinator_main -->|"updates tree"| node_src_coordinator_tree
node_src_coordinator_main -->|"applies policy"| node_src_coordinator_policies
node_src_coordinator_main -->|"dispatches tools"| node_src_coordinator_tools
node_src_coordinator_main -->|"emits events"| node_src_events_bus
node_src_coordinator_tools -->|"runs experiments"| node_src_executor_main
node_src_coordinator_tools -->|"reuses primitives"| node_src_core_tools
node_src_executor_main -->|"edits and runs"| node_src_core_tools
node_src_executor_main -->|"calls model"| node_src_core_llm
node_src_executor_main -->|"reports evidence"| node_src_events_bus
node_src_events_bus -->|"feeds reports"| node_src_report
node_src_events_bus -->|"feeds live view"| node_src_dashboard
node_src_events_bus -->|"feeds UI"| node_src_webui
node_src_search_agent -->|"searches web"| node_src_core_tools
node_src_plugins -->|"extends behavior"| node_src_coordinator_tools
node_skills_bundle -->|"prompt assets"| node_src_core_agent
node_skills_bundle -->|"specializes roles"| node_src_coordinator_main
node_skills_bundle -->|"specializes roles"| node_src_executor_main
node_src_zoo -->|"packages tasks"| node_arbor_zoo
node_src_zoo -->|"uses tools"| node_src_core_tools
node_src_mcp -->|"exposes tools"| node_src_core_tools
node_docs -.->|"documents"| node_src_cli_app
node_project_page -.->|"showcases"| node_src_report

click node_src_cli_app "https://github.com/ruc-nlpir/arbor/blob/main/src/cli/app.py"
click node_src_cli_commands "https://github.com/ruc-nlpir/arbor/tree/main/src/cli/commands"
click node_src_cli_intake "https://github.com/ruc-nlpir/arbor/tree/main/src/cli/intake"
click node_src_cli_resume "https://github.com/ruc-nlpir/arbor/blob/main/src/cli/resume_picker.py"
click node_src_core_config "https://github.com/ruc-nlpir/arbor/blob/main/src/core/config_resolve.py"
click node_src_core_agent "https://github.com/ruc-nlpir/arbor/blob/main/src/core/agent.py"
click node_src_core_tools "https://github.com/ruc-nlpir/arbor/tree/main/src/core/tools"
click node_src_core_llm "https://github.com/ruc-nlpir/arbor/tree/main/src/core/llm"
click node_src_coordinator_main "https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/orchestrator.py"
click node_src_coordinator_tree "https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/idea_tree.py"
click node_src_coordinator_policies "https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/contamination.py"
click node_src_coordinator_tools "https://github.com/ruc-nlpir/arbor/tree/main/src/coordinator/tools"
click node_src_executor_main "https://github.com/ruc-nlpir/arbor/blob/main/src/executor/main.py"
click node_src_events_bus "https://github.com/ruc-nlpir/arbor/blob/main/src/events/bus.py"
click node_src_report "https://github.com/ruc-nlpir/arbor/blob/main/src/report/generator.py"
click node_src_dashboard "https://github.com/ruc-nlpir/arbor/blob/main/src/dashboard.py"
click node_src_webui "https://github.com/ruc-nlpir/arbor/blob/main/src/webui/server.py"
click node_src_search_agent "https://github.com/ruc-nlpir/arbor/blob/main/src/search_agent/agent.py"
click node_src_plugins "https://github.com/ruc-nlpir/arbor/tree/main/src/plugins"
click node_skills_bundle "https://github.com/ruc-nlpir/arbor/tree/main/skills"
click node_src_mcp "https://github.com/ruc-nlpir/arbor/blob/main/src/mcp/server.py"
click node_src_zoo "https://github.com/ruc-nlpir/arbor/tree/main/src/zoo"
click node_arbor_zoo "https://github.com/ruc-nlpir/arbor/tree/main/arbor-zoo"
click node_docs "https://github.com/ruc-nlpir/arbor/tree/main/docs"
click node_project_page "https://github.com/ruc-nlpir/arbor/tree/main/project_page"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_cli_app,node_src_cli_commands,node_src_cli_intake,node_src_cli_resume,node_src_zoo,node_arbor_zoo toneBlue
class node_src_core_config,node_src_core_agent,node_src_core_tools,node_src_core_llm,node_docs,node_project_page toneAmber
class node_src_coordinator_main,node_src_coordinator_tree,node_src_coordinator_policies,node_src_coordinator_tools toneMint
class node_src_executor_main toneRose
class node_src_events_bus,node_src_report,node_src_dashboard,node_src_webui toneIndigo
class node_src_search_agent,node_src_plugins,node_skills_bundle,node_src_mcp toneTeal
```

## Components

### CLI

Operator-facing entrypoints and run control

- [arbor CLI](https://github.com/ruc-nlpir/arbor/blob/main/src/cli/app.py): entrypoint · Main command shell and startup wiring · `src/cli/app.py`
- [Commands](https://github.com/ruc-nlpir/arbor/tree/main/src/cli/commands): command layer · Subcommands for setup, run, login, replay, export, web, and diagnostics · `src/cli/commands`
- [Intake flow](https://github.com/ruc-nlpir/arbor/tree/main/src/cli/intake): interactive setup · Conversation capture, goal intake, and research contract setup · `src/cli/intake`
- [Resume tools](https://github.com/ruc-nlpir/arbor/blob/main/src/cli/resume_picker.py): session control · Resume picker, run state, and post-run handling · `src/cli/resume_picker.py`

### Core runtime

Shared agent, config, tool, and model infrastructure

- [Config system](https://github.com/ruc-nlpir/arbor/blob/main/src/core/config_resolve.py): configuration · Schema, resolution, CLI config, and user settings · `src/core/config_resolve.py`
- [Agent core](https://github.com/ruc-nlpir/arbor/blob/main/src/core/agent.py): agent runtime · Shared agent abstractions and execution scaffolding · `src/core/agent.py`
- [Tooling](https://github.com/ruc-nlpir/arbor/tree/main/src/core/tools): toolkit · File, shell, skill, executor, and web tools · `src/core/tools`
- [LLM adapters](https://github.com/ruc-nlpir/arbor/tree/main/src/core/llm): model boundary · Provider adapters for Anthropic, OpenAI, and compatible backends · `src/core/llm`

### Coordinator

Planning, idea tree, and policy loop

- [Coordinator](https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/orchestrator.py): decision engine · Owns arbor cycle orchestration and high-level policy · `src/coordinator/orchestrator.py`
- [Idea tree](https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/idea_tree.py): state model · Branching hypothesis tree and checkpoints · `src/coordinator/idea_tree.py`
- [Policies](https://github.com/ruc-nlpir/arbor/blob/main/src/coordinator/contamination.py): safety policy · Contamination, convergence, pruning, and human-in-the-loop checks · `src/coordinator/contamination.py`
- [Coord tools](https://github.com/ruc-nlpir/arbor/tree/main/src/coordinator/tools): coord tools · Protected file access, git/worktree ops, executor dispatch, and findings capture · `src/coordinator/tools`

### Executor

Isolated experiment runner

- [Executor](https://github.com/ruc-nlpir/arbor/blob/main/src/executor/main.py): worker runtime · Isolated hypothesis execution and evaluation worker · `src/executor/main.py`

### Events and reporting

Run artifacts, replay, and presentation layers

- [Event bus](https://github.com/ruc-nlpir/arbor/blob/main/src/events/bus.py): event system · Structured event emission, payloads, and subscribers · `src/events/bus.py`
- [Reports](https://github.com/ruc-nlpir/arbor/blob/main/src/report/generator.py): rendering · Human-readable summaries and run reports · `src/report/generator.py`
- [Dashboard](https://github.com/ruc-nlpir/arbor/blob/main/src/dashboard.py): ui server · Session browser and web dashboard entrypoint · `src/dashboard.py`
- [Web UI](https://github.com/ruc-nlpir/arbor/blob/main/src/webui/server.py): frontend server · Static session viewer and browser-based UI · `src/webui/server.py`

### Extensions

Plugins, skills, search, and MCP integration

- [Search agent](https://github.com/ruc-nlpir/arbor/blob/main/src/search_agent/agent.py): novelty search · Research and related-work search orchestration · `src/search_agent/agent.py`
- [Plugins](https://github.com/ruc-nlpir/arbor/tree/main/src/plugins): yaml extensions · Domain-specific plugin manifests and behavior hooks · `src/plugins`
- [Skills](https://github.com/ruc-nlpir/arbor/tree/main/skills): prompt assets · Reusable markdown skills for coordinator, executor, search, and setup flows · `skills`
- [MCP server](https://github.com/ruc-nlpir/arbor/blob/main/src/mcp/server.py): integration boundary · Exposes deterministic Arbor tools to host models · `src/mcp/server.py`

### Benchmark zoo

Benchmark packaging and task scaffolding

- [Benchmark zoo](https://github.com/ruc-nlpir/arbor/tree/main/src/zoo): benchmark packager · Acquire, scaffold, pack, verify, and collect benchmark tasks · `src/zoo`
- [Zoo assets](https://github.com/ruc-nlpir/arbor/tree/main/arbor-zoo): benchmark library · Packaged benchmark templates and example tasks · `arbor-zoo`

### Public surfaces

Docs and marketing site

- [Docs](https://github.com/ruc-nlpir/arbor/tree/main/docs): documentation · MkDocs reference, guides, and architecture notes · `docs`
- [Website](https://github.com/ruc-nlpir/arbor/tree/main/project_page): marketing site · React/Vite public site and demo assets · `project_page`

## Connections

- arbor CLI → Commands: dispatches · CLI entrypoint routes to command handlers
- arbor CLI → Intake flow: starts intake · Interactive setup collects research goals
- arbor CLI → Config system: resolves config · Startup loads and merges user and run configuration
- arbor CLI → Coordinator: launches loop · CLI run starts the coordinator orchestration cycle
- arbor CLI → Executor: spawns workers · Execution commands invoke isolated executor runs
- arbor CLI → Dashboard: opens dashboard · Operator can launch a local session dashboard
- arbor CLI → MCP server: serves tools · CLI can expose Arbor over MCP for host-driven use
- Coordinator → Idea tree: updates tree · Coordinator reads and mutates the idea tree
- Coordinator → Policies: applies policy · Decision loop consults pruning, contamination, and HITL checks
- Coordinator → Coord tools: dispatches tools · Coordinator uses protected repo and executor-control tools
- Coordinator → Event bus: emits events · Coordinator publishes structured lifecycle events
- Coord tools → Executor: runs experiments · Coordinator launches executor worktree jobs
- Coord tools → Tooling: reuses primitives · Coordination tools build on shared file and shell abstractions
- Executor → Tooling: edits and runs · Executor uses shared tool primitives to modify code and run tasks
- Executor → LLM adapters: calls model · Executor agents rely on model adapters for reasoning and actions
- Executor → Event bus: reports evidence · Executor emits structured results and evaluation events
- Event bus → Reports: feeds reports · Event stream becomes run summaries and reports
- Event bus → Dashboard: feeds live view · Dashboard consumes session events and snapshots
- Event bus → Web UI: feeds UI · Web UI renders stored session artifacts
- Search agent → Tooling: searches web · Search flows use web tools and related-work retrieval
- Plugins → Coord tools: extends behavior · Plugins add domain-specific orchestration and evaluation hooks
- Skills → Agent core: prompt assets · Skills guide shared agent behavior across roles
- Skills → Coordinator: specializes roles · Coordinator-oriented skills define planner behaviors
- Skills → Executor: specializes roles · Executor-oriented skills define worker behaviors
- Benchmark zoo → Zoo assets: packages tasks · Zoo subsystem manages benchmark templates and example bundles
- Benchmark zoo → Tooling: uses tools · Benchmark acquisition and verification rely on shared tooling
- MCP server → Tooling: exposes tools · MCP server publishes deterministic tools to external hosts
- Docs → arbor CLI: documents · Docs explain the CLI and user workflows
- Website → Reports: showcases · Public site mirrors reports and demo outputs
