# harbor-framework/harbor architecture

> Architecture diagram of the GitHub repository harbor-framework/harbor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harbor-framework/harbor
- Repository: https://github.com/harbor-framework/harbor
- Video tour (about a minute): https://gitdiagram.com/harbor-framework/harbor/video

## Overview

Harbor is a framework for evaluating agents and language models in sandboxed benchmark environments, creating benchmarks, running parallel experiments, and generating RL rollouts. Its principal workflow is to select a dataset and agent/model, execute tasks in an environment, score results, and inspect or upload runs. The graph preserves these documented capabilities and the registry, viewer, and upload subsystems. Evidence is limited to the README, the source index, and sampled excerpts; unverified runtime links are omitted or marked dashed.

Main entry points: [`src/harbor/cli`](https://github.com/harbor-framework/harbor/tree/main/src/harbor/cli) (Harbor CLI), [`src/harbor/viewer/server.py`](https://github.com/harbor-framework/harbor/blob/main/src/harbor/viewer/server.py) (Viewer API), [`src/harbor/registry/client/json.py`](https://github.com/harbor-framework/harbor/blob/main/src/harbor/registry/client/json.py) (Dataset registry), [`src/harbor/agents`](https://github.com/harbor-framework/harbor/tree/main/src/harbor/agents) (Agent adapters)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_run["Evaluation runs"]
  node_cli["Harbor CLI"]
  node_agent["Agent adapters"]
end

subgraph group_datasets["Datasets and tasks"]
  node_taskconfig["Task configuration<br/>[config.py]"]
  node_benchadapters["Benchmark adapters"]
  node_registrycli["Dataset commands<br/>[datasets.py]"]
  node_registry["Dataset registry<br/>[json.py]"]
end

subgraph group_infra["Runtime infrastructure"]
  node_environment["Sandbox environments"]
  node_providers["Environment providers"]
end

subgraph group_results["Results and analysis"]
  node_scoring["Scoring and metrics"]
  node_rollouts["Rollout generation"]
  node_runresults[("Run results")]
  node_viewer["Viewer API<br/>[server.py]"]
  node_viewerui["Viewer UI"]
  node_upload["Run upload"]
end

node_user(("Engineer"))
node_hub[("Harbor Hub")]
node_models["Model services"]

node_user -->|"starts run"| node_cli
node_cli -.->|"selects dataset"| node_registry
node_registrycli -->|"lists datasets"| node_registry
node_registry -.->|"describes datasets"| node_benchadapters
node_cli -.->|"selects agent"| node_agent
node_cli -.->|"configures tasks"| node_taskconfig
node_benchadapters -.->|"defines task inputs"| node_taskconfig
node_agent -.->|"requests model output"| node_models
node_cli -->|"launches tasks"| node_environment
node_environment -->|"runs on provider"| node_providers
node_cli -.->|"evaluates outputs"| node_scoring
node_cli -->|"generates rollouts"| node_rollouts
node_cli -.->|"records run"| node_runresults
node_viewerui -.->|"requests run data"| node_viewer
node_viewer -.->|"reads run data"| node_runresults
node_viewer -->|"checks upload status"| node_hub
node_upload -->|"transfers run files"| node_hub
node_user -.->|"inspects results"| node_viewerui

click node_cli "https://github.com/harbor-framework/harbor/tree/main/src/harbor/cli"
click node_agent "https://github.com/harbor-framework/harbor/tree/main/src/harbor/agents"
click node_taskconfig "https://github.com/harbor-framework/harbor/blob/main/src/harbor/models/task/config.py"
click node_benchadapters "https://github.com/harbor-framework/harbor/tree/main/adapters"
click node_registrycli "https://github.com/harbor-framework/harbor/blob/main/src/harbor/cli/datasets.py"
click node_registry "https://github.com/harbor-framework/harbor/blob/main/src/harbor/registry/client/json.py"
click node_environment "https://github.com/harbor-framework/harbor/tree/main/src/harbor"
click node_scoring "https://github.com/harbor-framework/harbor/tree/main/adapters"
click node_rollouts "https://github.com/harbor-framework/harbor/tree/main/src/harbor"
click node_runresults "https://github.com/harbor-framework/harbor/tree/main/src/harbor/db"
click node_viewer "https://github.com/harbor-framework/harbor/blob/main/src/harbor/viewer/server.py"
click node_viewerui "https://github.com/harbor-framework/harbor/tree/main/apps/viewer/app"
click node_upload "https://github.com/harbor-framework/harbor/tree/main/src/harbor/upload"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_agent toneBlue
class node_taskconfig,node_benchadapters,node_registrycli,node_registry,node_hub toneAmber
class node_environment,node_providers toneMint
class node_scoring,node_rollouts,node_runresults,node_viewer,node_viewerui,node_upload toneRose
class node_user,node_models toneIndigo
```

## Components

### Evaluation runs

- [Harbor CLI](https://github.com/harbor-framework/harbor/tree/main/src/harbor/cli): `src/harbor/cli`
- [Agent adapters](https://github.com/harbor-framework/harbor/tree/main/src/harbor/agents): `src/harbor/agents`

### Datasets and tasks

- [Task configuration](https://github.com/harbor-framework/harbor/blob/main/src/harbor/models/task/config.py): `src/harbor/models/task/config.py`
- [Benchmark adapters](https://github.com/harbor-framework/harbor/tree/main/adapters): `adapters`
- [Dataset commands](https://github.com/harbor-framework/harbor/blob/main/src/harbor/cli/datasets.py): `src/harbor/cli/datasets.py`
- [Dataset registry](https://github.com/harbor-framework/harbor/blob/main/src/harbor/registry/client/json.py): `src/harbor/registry/client/json.py`

### Runtime infrastructure

- [Sandbox environments](https://github.com/harbor-framework/harbor/tree/main/src/harbor): `src/harbor`
- **Environment providers**

### Results and analysis

- [Scoring and metrics](https://github.com/harbor-framework/harbor/tree/main/adapters): `adapters`
- [Rollout generation](https://github.com/harbor-framework/harbor/tree/main/src/harbor): `src/harbor`
- [Run results](https://github.com/harbor-framework/harbor/tree/main/src/harbor/db): `src/harbor/db`
- [Viewer API](https://github.com/harbor-framework/harbor/blob/main/src/harbor/viewer/server.py): `src/harbor/viewer/server.py`
- [Viewer UI](https://github.com/harbor-framework/harbor/tree/main/apps/viewer/app): `apps/viewer/app`
- [Run upload](https://github.com/harbor-framework/harbor/tree/main/src/harbor/upload): `src/harbor/upload`

### Other components

- **Engineer**
- **Harbor Hub**
- **Model services**

## Connections

- Engineer → Harbor CLI: starts run (evidence: `README.md`)
- Harbor CLI → Dataset registry: selects dataset
- Dataset commands → Dataset registry: lists datasets (evidence: `src/harbor/cli/datasets.py`)
- Dataset registry → Benchmark adapters: describes datasets (evidence: `adapters/bird_bench/src/bird_bench/main.py`)
- Harbor CLI → Agent adapters: selects agent
- Harbor CLI → Task configuration: configures tasks
- Benchmark adapters → Task configuration: defines task inputs (evidence: `adapters/frontier-cs-algorithm/src/frontier_cs_algorithm/adapter.py`)
- Agent adapters → Model services: requests model output
- Harbor CLI → Sandbox environments: launches tasks (evidence: `README.md`)
- Sandbox environments → Environment providers: runs on provider (evidence: `README.md`)
- Harbor CLI → Scoring and metrics: evaluates outputs (evidence: `adapters/bird_bench/src/bird_bench/main.py`)
- Harbor CLI → Rollout generation: generates rollouts (evidence: `README.md`)
- Harbor CLI → Run results: records run (evidence: `src/harbor/cli/datasets.py`)
- Viewer UI → Viewer API: requests run data
- Viewer API → Run results: reads run data (evidence: `src/harbor/viewer/server.py`)
- Viewer API → Harbor Hub: checks upload status (evidence: `src/harbor/viewer/server.py`)
- Run upload → Harbor Hub: transfers run files (evidence: `src/harbor/upload/storage.py`)
- Engineer → Viewer UI: inspects results
