# microsoft/tokeneconomizer architecture

> Architecture diagram of the GitHub repository microsoft/tokeneconomizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/tokeneconomizer
- Repository: https://github.com/microsoft/tokeneconomizer

## Overview

TokenEconomizer is a repository-local plugin for Claude Code, Codex, Copilot CLI, and Pi. Agent hooks observe sessions; finished-session traces update repository state and inform later sessions through micro-skills, tool pruning, and failure-loop intervention. The README also documents cache-aware cost accounting and benchmark experiments. Sampled excerpts show the live controller and folding/state mechanisms; several host and benchmark links below rely on README claims or SOURCE INDEX references rather than complete source excerpts.

Main entry points: [`tokeneconomizer/plugin/controller.py`](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/controller.py) (Live controller), [`tokeneconomizer/plugin/fold.py`](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/fold.py) (Session folding), [`tokeneconomizer/plugin/context.py`](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/context.py) (Repository context), [`experiments/te_harness/swebench.py`](https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_harness/swebench.py) (Benchmark execution)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_integration["Agent integration"]
  node_hooks["Host hooks<br/>[hook.py]"]
  node_pi_ext["Pi extension<br/>[tokeneconomizer.ts]"]
  node_host_adapters["Host adapters<br/>[hosts.py]"]
end

subgraph group_runtime["Live runtime"]
  node_controller["Live controller<br/>[controller.py]"]
  node_context["Repository context<br/>[context.py]"]
  node_repo_state["Repository identity<br/>[repos.py]"]
  node_state_paths["State paths<br/>[paths.py]"]
  node_store[("Local state store<br/>[store.py]")]
  node_cli["Plugin commands<br/>[cli.py]"]
end

subgraph group_learning["Session learning"]
  node_trace["Session trace readers<br/>[calls.py]"]
  node_observe["Environment observations<br/>[observe.py]"]
  node_micro_skills["Micro-skills<br/>[lifecycle.py]"]
  node_fold["Session folding<br/>[fold.py]"]
  node_tool_prune["Tool pruning<br/>[stage.py]"]
  node_denylist["Agent settings<br/>[denylist.py]"]
  node_loop_controller["Loop intervention<br/>[controller.py]"]
  node_loop_state["Loop signals<br/>[state.py]"]
end

subgraph group_accounting["Cost and evaluation"]
  node_pricing["Cache-aware pricing<br/>[pricing.py]"]
  node_experiments["Benchmark workloads<br/>[workloads.py]"]
  node_bench_runner["Benchmark execution<br/>[swebench.py]"]
  node_codex_spiral["Codex spiral recovery<br/>[codex_spiral.py]"]
end

node_developer(("Developer"))
node_agents(("Coding agents"))

node_developer -->|"runs"| node_agents
node_agents -->|"fires hooks"| node_hooks
node_agents -->|"loads extension"| node_pi_ext
node_hooks -.->|"dispatches events"| node_controller
node_pi_ext -.->|"integrates hooks"| node_controller
node_controller -->|"builds context"| node_context
node_context -->|"resolves repository"| node_repo_state
node_context -->|"selects paths"| node_state_paths
node_context -->|"opens state"| node_store
node_controller -.->|"adapts host"| node_host_adapters
node_controller -->|"folds sessions"| node_fold
node_controller -->|"checks turns"| node_loop_controller
node_controller -->|"dispatches commands"| node_cli
node_fold -->|"reads traces"| node_trace
node_fold -->|"learns facts"| node_micro_skills
node_fold -->|"tracks usage"| node_tool_prune
node_fold -.->|"updates state"| node_store
node_micro_skills -->|"classifies evidence"| node_observe
node_tool_prune -->|"applies pruning"| node_denylist
node_denylist -->|"changes settings"| node_agents
node_loop_controller -->|"folds tool calls"| node_loop_state
node_loop_controller -->|"redirects once"| node_agents
node_agents -->|"produces session logs"| node_trace
node_experiments -.->|"supplies workloads"| node_bench_runner
node_bench_runner -->|"measures cost"| node_pricing
node_bench_runner -.->|"runs Codex recovery"| node_codex_spiral

click node_hooks "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/hook.py"
click node_pi_ext "https://github.com/microsoft/tokeneconomizer/blob/main/plugin/pi/tokeneconomizer.ts"
click node_host_adapters "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/hosts.py"
click node_controller "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/controller.py"
click node_context "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/context.py"
click node_repo_state "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/repos.py"
click node_state_paths "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/paths.py"
click node_store "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/store.py"
click node_cli "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/cli.py"
click node_trace "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/trace/calls.py"
click node_observe "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/microskills/observe.py"
click node_micro_skills "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/microskills/lifecycle.py"
click node_fold "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/fold.py"
click node_tool_prune "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/toolprune/stage.py"
click node_denylist "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/denylist.py"
click node_loop_controller "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/controller.py"
click node_loop_state "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/loopguard/state.py"
click node_pricing "https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/pricing.py"
click node_experiments "https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_bench/workloads.py"
click node_bench_runner "https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_harness/swebench.py"
click node_codex_spiral "https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_bench/codex_spiral.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_hooks,node_pi_ext,node_host_adapters toneBlue
class node_controller,node_context,node_repo_state,node_state_paths,node_store,node_cli toneAmber
class node_trace,node_observe,node_micro_skills,node_fold,node_tool_prune,node_denylist,node_loop_controller,node_loop_state toneMint
class node_pricing,node_experiments,node_bench_runner,node_codex_spiral toneRose
class node_developer,node_agents toneIndigo
```

## Components

### Agent integration

- [Host hooks](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/hook.py): `tokeneconomizer/plugin/hook.py`
- [Pi extension](https://github.com/microsoft/tokeneconomizer/blob/main/plugin/pi/tokeneconomizer.ts): `plugin/pi/tokeneconomizer.ts`
- [Host adapters](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/hosts.py): `tokeneconomizer/plugin/hosts.py`

### Live runtime

- [Live controller](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/controller.py): `tokeneconomizer/plugin/controller.py`
- [Repository context](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/context.py): `tokeneconomizer/plugin/context.py`
- [Repository identity](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/repos.py): `tokeneconomizer/plugin/repos.py`
- [State paths](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/paths.py): `tokeneconomizer/plugin/paths.py`
- [Local state store](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/store.py): `tokeneconomizer/plugin/store.py`
- [Plugin commands](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/cli.py): `tokeneconomizer/plugin/cli.py`

### Session learning

- [Session trace readers](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/trace/calls.py): `tokeneconomizer/trace/calls.py`
- [Environment observations](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/microskills/observe.py): `tokeneconomizer/microskills/observe.py`
- [Micro-skills](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/microskills/lifecycle.py): `tokeneconomizer/microskills/lifecycle.py`
- [Session folding](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/fold.py): `tokeneconomizer/plugin/fold.py`
- [Tool pruning](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/toolprune/stage.py): `tokeneconomizer/toolprune/stage.py`
- [Agent settings](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/denylist.py): `tokeneconomizer/plugin/denylist.py`
- [Loop intervention](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/plugin/controller.py): `tokeneconomizer/plugin/controller.py`
- [Loop signals](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/loopguard/state.py): `tokeneconomizer/loopguard/state.py`

### Cost and evaluation

- [Cache-aware pricing](https://github.com/microsoft/tokeneconomizer/blob/main/tokeneconomizer/pricing.py): `tokeneconomizer/pricing.py`
- [Benchmark workloads](https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_bench/workloads.py): `experiments/te_bench/workloads.py`
- [Benchmark execution](https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_harness/swebench.py): `experiments/te_harness/swebench.py`
- [Codex spiral recovery](https://github.com/microsoft/tokeneconomizer/blob/main/experiments/te_bench/codex_spiral.py): `experiments/te_bench/codex_spiral.py`

### Other components

- **Developer**
- **Coding agents**

## Connections

- Developer → Coding agents: runs
- Coding agents → Host hooks: fires hooks
- Coding agents → Pi extension: loads extension
- Host hooks → Live controller: dispatches events
- Pi extension → Live controller: integrates hooks
- Live controller → Repository context: builds context (evidence: `tokeneconomizer/plugin/context.py`)
- Repository context → Repository identity: resolves repository (evidence: `tokeneconomizer/plugin/context.py`)
- Repository context → State paths: selects paths (evidence: `tokeneconomizer/plugin/context.py`)
- Repository context → Local state store: opens state (evidence: `tokeneconomizer/plugin/context.py`)
- Live controller → Host adapters: adapts host (evidence: `tokeneconomizer/plugin/controller.py`)
- Live controller → Session folding: folds sessions (evidence: `tokeneconomizer/plugin/controller.py`)
- Live controller → Loop intervention: checks turns (evidence: `tokeneconomizer/plugin/controller.py`)
- Live controller → Plugin commands: dispatches commands (evidence: `tokeneconomizer/plugin/cli.py`)
- Session folding → Session trace readers: reads traces (evidence: `tokeneconomizer/plugin/fold.py`)
- Session folding → Micro-skills: learns facts (evidence: `tokeneconomizer/plugin/fold.py`)
- Session folding → Tool pruning: tracks usage (evidence: `tokeneconomizer/plugin/fold.py`)
- Session folding → Local state store: updates state
- Micro-skills → Environment observations: classifies evidence (evidence: `tokeneconomizer/microskills/envfacts.py`)
- Tool pruning → Agent settings: applies pruning (evidence: `tokeneconomizer/plugin/denylist.py`)
- Agent settings → Coding agents: changes settings (evidence: `tokeneconomizer/plugin/denylist.py`)
- Loop intervention → Loop signals: folds tool calls
- Loop intervention → Coding agents: redirects once
- Coding agents → Session trace readers: produces session logs
- Benchmark workloads → Benchmark execution: supplies workloads
- Benchmark execution → Cache-aware pricing: measures cost (evidence: `experiments/te_harness/swebench.py`)
- Benchmark execution → Codex spiral recovery: runs Codex recovery (evidence: `experiments/te_harness/swebench.py`)