# guybass/agent-eval-flow architecture

> Architecture diagram of the GitHub repository guybass/agent-eval-flow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/guybass/agent-eval-flow
- Repository: https://github.com/guybass/agent-eval-flow

## Overview

Agent Eval Flow evaluates agent runs and configuration changes from retained or newly captured evidence. The principal workflow is to define candidates and checks, run agents or import captures through adapters, evaluate evidence, inspect HTML reports, and compare or select candidate results. Separate assessment workflows cover configuration and observed runtime behavior; Toolscore adds optional tool-call diagnostics. The graph preserves these documented capabilities. Some implementation paths are represented at their exact module paths; relationships not shown in the excerpts or source references are omitted.

Main entry points: [`src/agent_eval_flow/pipeline/assessment.py`](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/pipeline/assessment.py) (Configuration assessment), [`src/agent_eval_flow/execution/runner.py`](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/execution/runner.py) (Run dispatcher), [`src/agent_eval_flow/results/query.py`](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/results/query.py) (Result queries), [`src/agent_eval_flow/evaluation/engine.py`](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/engine.py) (Evaluation engine)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Workflow"]
  node_pipeline["Evaluation pipeline<br/>[api.py]"]
  node_assessment["Configuration assessment<br/>[assessment.py]"]
end

subgraph group_capture["Capture and adapters"]
  node_runner["Run dispatcher<br/>[runner.py]"]
  node_capture_buffer["Capture receipts<br/>[capture.py]"]
  node_adapter["Runtime adapters"]
  node_common_adapter["Adapter utilities<br/>[common.py]"]
end

subgraph group_model["Evidence model"]
  node_records["Run records<br/>[records.py]"]
  node_assessment_records["Assessment records<br/>[assessment.py]"]
  node_validation["Assessment validation"]
  node_identity["Stable identities<br/>[identity.py]"]
end

subgraph group_evaluation["Evaluation and results"]
  node_engine["Evaluation engine<br/>[engine.py]"]
  node_runtime_checks["Runtime checks<br/>[runtime_checks.py]"]
  node_toolscore_eval["Toolscore evaluator<br/>[toolscore.py]"]
  node_query["Result queries<br/>[query.py]"]
  node_selection["Candidate selection"]
end

subgraph group_output["Reports and storage"]
  node_html_report["HTML reports<br/>[html.py]"]
  node_tool_report["Toolscore report<br/>[toolscore.py]"]
  node_artifact_store[("Evidence artifacts<br/>[artifacts.py]")]
  node_manifest_store[("Result manifests<br/>[manifests.py]")]
end

node_developer(("Developer"))

node_developer -->|"defines evaluation"| node_pipeline
node_developer -->|"assesses configuration"| node_assessment
node_pipeline -->|"runs agents"| node_runner
node_pipeline -->|"evaluates captures"| node_engine
node_assessment -->|"captures behavior"| node_runner
node_assessment -->|"evaluates checks"| node_engine
node_runner -->|"records receipts"| node_capture_buffer
node_runner -->|"dispatches runs"| node_adapter
node_adapter -->|"uses utilities"| node_common_adapter
node_common_adapter -->|"stores evidence"| node_artifact_store
node_runner -->|"returns run records"| node_records
node_records -->|"fingerprints records"| node_identity
node_assessment_records -->|"validates records"| node_validation
node_assessment -->|"checks behavior"| node_runtime_checks
node_assessment -->|"selects candidates"| node_selection
node_engine -->|"produces queryable results"| node_query
node_toolscore_eval -->|"creates diagnostics"| node_tool_report
node_query -->|"supplies report data"| node_html_report
node_query -->|"supports selection"| node_selection
node_html_report -->|"presents findings"| node_developer
node_engine -.->|"reads evidence"| node_artifact_store
node_query -.->|"loads saved results"| node_manifest_store

click node_pipeline "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/pipeline/api.py"
click node_assessment "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/pipeline/assessment.py"
click node_runner "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/execution/runner.py"
click node_capture_buffer "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/execution/capture.py"
click node_adapter "https://github.com/guybass/agent-eval-flow/tree/main/src/agent_eval_flow/adapters"
click node_common_adapter "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/adapters/common.py"
click node_records "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/records.py"
click node_assessment_records "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/assessment.py"
click node_validation "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/assessment_validation.py"
click node_identity "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/identity.py"
click node_engine "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/engine.py"
click node_runtime_checks "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/runtime_checks.py"
click node_toolscore_eval "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/toolscore.py"
click node_query "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/results/query.py"
click node_selection "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/results/assessment_selection.py"
click node_html_report "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/reporting/html.py"
click node_tool_report "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/reporting/toolscore.py"
click node_artifact_store "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/storage/artifacts.py"
click node_manifest_store "https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/storage/manifests.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_assessment toneBlue
class node_runner,node_capture_buffer,node_adapter,node_common_adapter toneAmber
class node_records,node_assessment_records,node_validation,node_identity toneMint
class node_engine,node_runtime_checks,node_toolscore_eval,node_query,node_selection toneRose
class node_html_report,node_tool_report,node_artifact_store,node_manifest_store,node_developer toneIndigo
```

## Components

### Workflow

- [Evaluation pipeline](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/pipeline/api.py): `src/agent_eval_flow/pipeline/api.py`
- [Configuration assessment](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/pipeline/assessment.py): `src/agent_eval_flow/pipeline/assessment.py`

### Capture and adapters

- [Run dispatcher](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/execution/runner.py): `src/agent_eval_flow/execution/runner.py`
- [Capture receipts](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/execution/capture.py): `src/agent_eval_flow/execution/capture.py`
- [Runtime adapters](https://github.com/guybass/agent-eval-flow/tree/main/src/agent_eval_flow/adapters): `src/agent_eval_flow/adapters`
- [Adapter utilities](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/adapters/common.py): `src/agent_eval_flow/adapters/common.py`

### Evidence model

- [Run records](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/records.py): `src/agent_eval_flow/objects/records.py`
- [Assessment records](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/assessment.py): `src/agent_eval_flow/objects/assessment.py`
- [Assessment validation](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/assessment_validation.py): `src/agent_eval_flow/objects/assessment_validation.py`
- [Stable identities](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/objects/identity.py): `src/agent_eval_flow/objects/identity.py`

### Evaluation and results

- [Evaluation engine](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/engine.py): `src/agent_eval_flow/evaluation/engine.py`
- [Runtime checks](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/runtime_checks.py): `src/agent_eval_flow/evaluation/runtime_checks.py`
- [Toolscore evaluator](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/evaluation/toolscore.py): `src/agent_eval_flow/evaluation/toolscore.py`
- [Result queries](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/results/query.py): `src/agent_eval_flow/results/query.py`
- [Candidate selection](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/results/assessment_selection.py): `src/agent_eval_flow/results/assessment_selection.py`

### Reports and storage

- [HTML reports](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/reporting/html.py): `src/agent_eval_flow/reporting/html.py`
- [Toolscore report](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/reporting/toolscore.py): `src/agent_eval_flow/reporting/toolscore.py`
- [Evidence artifacts](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/storage/artifacts.py): `src/agent_eval_flow/storage/artifacts.py`
- [Result manifests](https://github.com/guybass/agent-eval-flow/blob/main/src/agent_eval_flow/storage/manifests.py): `src/agent_eval_flow/storage/manifests.py`

### Other components

- **Developer**

## Connections

- Developer → Evaluation pipeline: defines evaluation (evidence: `README.md`)
- Developer → Configuration assessment: assesses configuration (evidence: `README.md`)
- Evaluation pipeline → Run dispatcher: runs agents (evidence: `src/agent_eval_flow/pipeline/api.py`)
- Evaluation pipeline → Evaluation engine: evaluates captures (evidence: `src/agent_eval_flow/pipeline/api.py`)
- Configuration assessment → Run dispatcher: captures behavior (evidence: `src/agent_eval_flow/pipeline/assessment.py`)
- Configuration assessment → Evaluation engine: evaluates checks (evidence: `src/agent_eval_flow/pipeline/assessment.py`)
- Run dispatcher → Capture receipts: records receipts (evidence: `src/agent_eval_flow/execution/runner.py`)
- Run dispatcher → Runtime adapters: dispatches runs (evidence: `README.md`)
- Runtime adapters → Adapter utilities: uses utilities (evidence: `src/agent_eval_flow/adapters/openkritt.py`)
- Adapter utilities → Evidence artifacts: stores evidence (evidence: `src/agent_eval_flow/adapters/common.py`)
- Run dispatcher → Run records: returns run records (evidence: `src/agent_eval_flow/execution/runner.py`)
- Run records → Stable identities: fingerprints records (evidence: `src/agent_eval_flow/objects/records.py`)
- Assessment records → Assessment validation: validates records (evidence: `src/agent_eval_flow/objects/assessment.py`)
- Configuration assessment → Runtime checks: checks behavior (evidence: `src/agent_eval_flow/pipeline/assessment.py`)
- Configuration assessment → Candidate selection: selects candidates (evidence: `src/agent_eval_flow/pipeline/assessment.py`)
- Evaluation engine → Result queries: produces queryable results (evidence: `src/agent_eval_flow/evaluation/engine.py`)
- Toolscore evaluator → Toolscore report: creates diagnostics (evidence: `src/agent_eval_flow/evaluation/toolscore.py`)
- Result queries → HTML reports: supplies report data (evidence: `src/agent_eval_flow/reporting/html.py`)
- Result queries → Candidate selection: supports selection (evidence: `src/agent_eval_flow/results/assessment_selection.py`)
- HTML reports → Developer: presents findings (evidence: `README.md`)
- Evaluation engine → Evidence artifacts: reads evidence
- Result queries → Result manifests: loads saved results (evidence: `src/agent_eval_flow/storage/manifests.py`)
