# ja-zoe/tokeneyezed architecture

> Architecture diagram of the GitHub repository ja-zoe/tokeneyezed, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ja-zoe/tokeneyezed
- Repository: https://github.com/ja-zoe/tokeneyezed

## Overview

Tokeneyezed is a long-horizon coding-agent harness: it repeatedly plans and runs attempts toward a scored CommonMark-rendering goal, retaining useful history across restarts and agent handoffs. Its main workflow is orchestrated by the controller; Atlas-backed memory informs bounded briefs and records attempt results, while the observer gates tool actions and filters events. Separate product branches provide split-based scoring and retrieval evaluation, plus a read-only dashboard for comparing and inspecting runs. Wiring details are drawn from the README and sampled modules; unsampled implementation links are omitted where ownership is unclear.

Main entry points: [`src/tokeneyezed/controller/graph.py`](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/graph.py) (Run graph), [`src/tokeneyezed/observer/service.py`](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/service.py) (Observer service), [`src/tokeneyezed/data/retrieval_eval.py`](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/retrieval_eval.py) (Retrieval evaluation), [`src/tokeneyezed/controller/cli.py`](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/cli.py) (Run CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_controller["Run controller"]
  node_cli["Run CLI<br/>[cli.py]"]
  node_graph["Run graph<br/>[graph.py]"]
  node_planner["Attempt planner<br/>[planner.py]"]
  node_runner["Agent runner"]
end

subgraph group_memory["Atlas memory"]
  node_brief["Attempt brief<br/>[brief.py]"]
  node_goals[("Goal state<br/>[goals.py]")]
  node_ledger[("Attempt ledger<br/>[ledger.py]")]
  node_data_writes["Data writes<br/>[writes.py]"]
end

subgraph group_observer["Observer"]
  node_observer_shim["Hook shim<br/>[shim.py]"]
  node_observer_service["Observer service<br/>[service.py]"]
  node_gate["Pre-action gate<br/>[core.py]"]
  node_postcheck["Post-action review<br/>[postcheck.py]"]
  node_event_writer["Mongo event writer<br/>[storage.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_scorer["Split scorer<br/>[scorer.py]"]
  node_retrieval_eval["Retrieval evaluation<br/>[retrieval_eval.py]"]
end

subgraph group_dashboard["Dashboard"]
  node_dashboard_ui["Dashboard UI<br/>[app.js]"]
  node_dashboard_server["Dashboard API<br/>[server.py]"]
  node_dashboard_reads["Dashboard reads<br/>[dashboard_reads.py]"]
end

node_operator(("Operator"))
node_coding_agent(("Coding agent"))
node_task_spec["CommonMark spec"]
node_atlas[("MongoDB Atlas")]
node_voyage(("Voyage AI"))

node_operator -->|"starts or resumes"| node_cli
node_cli -->|"runs session"| node_graph
node_graph -->|"seeds goals"| node_goals
node_graph -->|"builds brief"| node_brief
node_brief -->|"reads memory"| node_atlas
node_graph -->|"plans attempt"| node_planner
node_planner -.->|"requests model"| node_atlas
node_graph -->|"runs attempt"| node_runner
node_runner -->|"launches"| node_coding_agent
node_coding_agent -->|"implements"| node_task_spec
node_graph -->|"scores attempt"| node_scorer
node_scorer -->|"validates against"| node_task_spec
node_graph -->|"updates ledger"| node_ledger
node_ledger -->|"persists attempts"| node_atlas
node_graph -->|"checkpoints run"| node_atlas
node_coding_agent -->|"emits hook events"| node_observer_shim
node_observer_shim -->|"posts events"| node_observer_service
node_observer_service -->|"checks pre-actions"| node_gate
node_observer_service -->|"checks post-actions"| node_postcheck
node_observer_service -->|"persists verdict"| node_event_writer
node_event_writer -->|"inserts event"| node_data_writes
node_data_writes -->|"writes event"| node_atlas
node_operator -->|"runs evaluation"| node_retrieval_eval
node_retrieval_eval -->|"queries attempts"| node_atlas
node_retrieval_eval -.->|"embeds queries"| node_voyage
node_operator -->|"views runs"| node_dashboard_ui
node_dashboard_ui -->|"requests API"| node_dashboard_server
node_dashboard_server -->|"routes reads"| node_dashboard_reads
node_dashboard_reads -->|"reads run data"| node_atlas

click node_cli "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/cli.py"
click node_graph "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/graph.py"
click node_planner "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/planner.py"
click node_runner "https://github.com/ja-zoe/tokeneyezed/tree/main/src/tokeneyezed/controller/runners"
click node_brief "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/brief.py"
click node_goals "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/goals.py"
click node_ledger "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/ledger.py"
click node_data_writes "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/writes.py"
click node_observer_shim "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/shim.py"
click node_observer_service "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/service.py"
click node_gate "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/core.py"
click node_postcheck "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/postcheck.py"
click node_event_writer "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/storage.py"
click node_scorer "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/eval/scorer.py"
click node_retrieval_eval "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/retrieval_eval.py"
click node_dashboard_ui "https://github.com/ja-zoe/tokeneyezed/blob/main/public/app.js"
click node_dashboard_server "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/dashboard/server.py"
click node_dashboard_reads "https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/dashboard_reads.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_graph,node_planner,node_runner toneBlue
class node_brief,node_goals,node_ledger,node_data_writes,node_atlas toneAmber
class node_observer_shim,node_observer_service,node_gate,node_postcheck,node_event_writer toneMint
class node_scorer,node_retrieval_eval toneRose
class node_dashboard_ui,node_dashboard_server,node_dashboard_reads,node_operator,node_coding_agent,node_task_spec,node_voyage toneIndigo
```

## Components

### Run controller

- [Run CLI](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/cli.py): `src/tokeneyezed/controller/cli.py`
- [Run graph](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/graph.py): `src/tokeneyezed/controller/graph.py`
- [Attempt planner](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/controller/planner.py): `src/tokeneyezed/controller/planner.py`
- [Agent runner](https://github.com/ja-zoe/tokeneyezed/tree/main/src/tokeneyezed/controller/runners): `src/tokeneyezed/controller/runners`

### Atlas memory

- [Attempt brief](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/brief.py): `src/tokeneyezed/data/brief.py`
- [Goal state](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/goals.py): `src/tokeneyezed/data/goals.py`
- [Attempt ledger](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/ledger.py): `src/tokeneyezed/data/ledger.py`
- [Data writes](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/writes.py): `src/tokeneyezed/data/writes.py`

### Observer

- [Hook shim](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/shim.py): `src/tokeneyezed/observer/shim.py`
- [Observer service](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/service.py): `src/tokeneyezed/observer/service.py`
- [Pre-action gate](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/core.py): `src/tokeneyezed/observer/core.py`
- [Post-action review](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/postcheck.py): `src/tokeneyezed/observer/postcheck.py`
- [Mongo event writer](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/observer/storage.py): `src/tokeneyezed/observer/storage.py`

### Evaluation

- [Split scorer](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/eval/scorer.py): `src/tokeneyezed/eval/scorer.py`
- [Retrieval evaluation](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/retrieval_eval.py): `src/tokeneyezed/data/retrieval_eval.py`

### Dashboard

- [Dashboard UI](https://github.com/ja-zoe/tokeneyezed/blob/main/public/app.js): `public/app.js`
- [Dashboard API](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/dashboard/server.py): `src/tokeneyezed/dashboard/server.py`
- [Dashboard reads](https://github.com/ja-zoe/tokeneyezed/blob/main/src/tokeneyezed/data/dashboard_reads.py): `src/tokeneyezed/data/dashboard_reads.py`

### Other components

- **Operator**
- **Coding agent**
- **CommonMark spec**
- **MongoDB Atlas**
- **Voyage AI**

## Connections

- Operator → Run CLI: starts or resumes
- Run CLI → Run graph: runs session
- Run graph → Goal state: seeds goals
- Run graph → Attempt brief: builds brief
- Attempt brief → MongoDB Atlas: reads memory
- Run graph → Attempt planner: plans attempt
- Attempt planner → MongoDB Atlas: requests model
- Run graph → Agent runner: runs attempt
- Agent runner → Coding agent: launches
- Coding agent → CommonMark spec: implements
- Run graph → Split scorer: scores attempt
- Split scorer → CommonMark spec: validates against
- Run graph → Attempt ledger: updates ledger
- Attempt ledger → MongoDB Atlas: persists attempts
- Run graph → MongoDB Atlas: checkpoints run
- Coding agent → Hook shim: emits hook events
- Hook shim → Observer service: posts events
- Observer service → Pre-action gate: checks pre-actions
- Observer service → Post-action review: checks post-actions
- Observer service → Mongo event writer: persists verdict
- Mongo event writer → Data writes: inserts event
- Data writes → MongoDB Atlas: writes event
- Operator → Retrieval evaluation: runs evaluation
- Retrieval evaluation → MongoDB Atlas: queries attempts
- Retrieval evaluation → Voyage AI: embeds queries
- Operator → Dashboard UI: views runs
- Dashboard UI → Dashboard API: requests API
- Dashboard API → Dashboard reads: routes reads
- Dashboard reads → MongoDB Atlas: reads run data