# antoinezambelli/forge architecture

> Architecture diagram of the GitHub repository antoinezambelli/forge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/antoinezambelli/forge
- Repository: https://github.com/antoinezambelli/forge

## Overview

1. **What this repository is**
   Forge is a Python 3.12+ package that sits between an agent loop and an LLM backend to make multi-step tool use more reliable. Its architecture centers on a shared inference pipeline that can be used directly in application code, embedded as middleware, or exposed as an OpenAI-compatible proxy.

2. **Top-level execution paths**
   There are three primary entry points:
   - **WorkflowRunner** for end-to-end agent workflows managed by Forge.
   - **Guardrails middleware** for projects that already own the loop but want Forge’s validation and recovery.
   - **Proxy server** for drop-in use behind OpenAI-compatible clients.
   These three paths share the same core inference, guardrail, context, and client layers.

3. **Core orchestration layer**
   `forge.core` is the center of the system. It defines the workflow model, message types, inference pipeline, step tracking, and the main agent loop. `run_inference()` is the shared “front half” that prepares context, validates/repairs responses, and hands back structured model output. `WorkflowRunner` builds the full cycle around that inference path, including tool execution and continuation.

4. **Workflow and tool model**
   Workflows are declarative: they define a name, description, tool set, required steps, a terminal tool, and a system prompt template. Each tool is represented by a `ToolDef` plus a `ToolSpec`, and tool calls are turned into structured `ToolCall` or `TextResponse` outputs. This is the boundary where model output becomes executable application behavior.

5. **Backend client abstraction**
   `forge.clients` isolates model-provider differences behind a common LLM client protocol. The repo supports:
   - **Ollama** with native function calling,
   - **llama-server / llama.cpp** with native function calling or prompt-injected mode,
   - **Llamafile** as a single-binary backend,
   - **Anthropic** as a hosted baseline.
   This layer is the outbound boundary from Forge to inference infrastructure.

6. **Context management subsystem**
   `forge.context` manages token budgets and compaction. `ContextManager` applies a chosen strategy such as sliding window or tiered compaction, while hardware detection informs budgeting decisions. This subsystem is critical because the repo is optimized for constrained local models and needs to preserve the important parts of multi-turn tool interactions within VRAM-aware limits.

7. **Guardrails and recovery**
   `forge.guardrails` is the reliability layer. It includes response validation, retry nudges, step enforcement, and error tracking. The guardrails are designed to catch malformed tool responses, enforce required workflow steps, and recover from common failure modes without exposing that complexity to the caller.

8. **Prompting and synthetic tools**
   `forge.prompts` contains templates for tool-injected prompting and for retry/step-enforcement nudges. `forge.tools.respond` provides the synthetic `respond` tool used by the proxy when text output must be coerced into tool-calling mode. This is a key architectural trick: the model is guided to emit structured intent, then Forge strips the synthetic tool from the final client-visible response.

9. **Proxy boundary**
   `forge.proxy` exposes an asyncio-based HTTP server that speaks OpenAI-compatible APIs. It converts incoming OpenAI-style messages into Forge messages, runs them through the same inference/guardrail pipeline, and streams responses back. The proxy can run in **external mode** against an already-running backend or **managed mode** where Forge launches the backend alongside the proxy.

10. **Shared runtime / server management**
    `forge.server` and related runtime helpers coordinate backend startup and budget mode selection. This layer is what lets the package manage local model servers in a predictable way, especially for proxy-managed deployments and workflows that need backend lifecycle control.

11. **SlotWorker and multi-agent access**
    `forge.core.slot_worker` adds a priority-queued access model to a shared inference slot. This is a boundary for multi-agent or multi-workflow systems that need one GPU-backed model instance shared across competing tasks, with preemption and scheduling handled centrally instead of by the caller.

12. **Evaluation and validation infrastructure**
    The `tests/eval` tree is a full evaluation harness, not just unit tests. It includes scenario definitions, batch evaluation, metrics, significance testing, and an HTML dashboard for reporting. This infrastructure is important architecturally because the project’s claims and design decisions are grounded in a 26-scenario eval suite, with separate baselines and reasoning-oriented tiers.

13. **Repository boundaries**
    `tests/unit` validates deterministic behavior for the core subsystems without requiring a live model backend. `tests/eval` exercises real backend integrations and model behavior. Documentation under `docs/` captures architecture decisions, workflow internals, model/backend setup, and published evaluation results; it functions as the design record for the system rather than just user docs.

14. **Architecture summary for a diagram**
    The clearest diagram is: application or proxy client on the left; Forge core in the middle; guardrails, context management, and workflow orchestration as the central control plane; client adapters on the outbound side; and one of several backends on the far right. Tool execution loops back from the workflow layer into the application-defined tools, while eval/test infrastructure sits alongside the runtime as a validation and regression boundary.

Main entry points: [`src/forge/clients/base.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py) (Client protocol), [`src/forge/core/inference.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/inference.py) (Inference), [`src/forge/guardrails/guardrails.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/guardrails.py) (Guardrails), [`src/forge/core/runner.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/runner.py) (WorkflowRunner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_core["Core runtime"]
  node_n_runner["WorkflowRunner<br/>orchestrator<br/>[runner.py]"]
  node_n_inference["Inference<br/>pipeline<br/>[inference.py]"]
  node_n_workflow["Workflow model<br/>schema<br/>[workflow.py]"]
  node_n_messages["Messages<br/>types<br/>[messages.py]"]
  node_n_steps["Steps<br/>state<br/>[steps.py]"]
  node_n_slot_worker["Slot worker<br/>scheduler<br/>[slot_worker.py]"]
end

subgraph group_g_io["I/O boundaries"]
  node_n_client_base["Client protocol<br/>adapter<br/>[base.py]"]
  node_n_client_ollama["Ollama<br/>backend adapter<br/>[ollama.py]"]
  node_n_client_llama["Llama server<br/>backend adapter<br/>[base.py]"]
  node_n_client_lf["Llamafile<br/>backend adapter<br/>[llamafile.py]"]
  node_n_client_anthropic["Anthropic<br/>backend adapter<br/>[anthropic.py]"]
  node_n_proxy["Proxy server<br/>http server<br/>[server.py]"]
  node_n_proxy_convert["Convert<br/>translation<br/>[convert.py]"]
  node_n_proxy_handler["Handler<br/>request flow<br/>[handler.py]"]
  node_n_server["Server mgmt<br/>runtime<br/>[server.py]"]
  node_n_respond["Respond tool<br/>synthetic tool<br/>[respond.py]"]
end

subgraph group_g_reliability["Reliability"]
  node_n_ctx_mgr["Context manager<br/>budgeting<br/>[manager.py]"]
  node_n_ctx_strat["Strategies<br/>compaction<br/>[strategies.py]"]
  node_n_hardware["Hardware<br/>detection<br/>[hardware.py]"]
  node_n_guardrails["Guardrails<br/>recovery<br/>[guardrails.py]"]
  node_n_validator["Validator<br/>validation"]
  node_n_enforcer["Step enforcer<br/>policy<br/>[step_enforcer.py]"]
  node_n_nudge["Nudges<br/>repair<br/>[nudge.py]"]
  node_n_prompts["Prompts<br/>templates<br/>[templates.py]"]
end

subgraph group_g_validation["Validation"]
  node_n_eval["Eval suite<br/>evaluation<br/>[eval_runner.py]"]
  node_n_dashboard["Dashboard<br/>reporting ui<br/>[App.tsx]"]
end

subgraph group_g_docs["Docs"]
  node_n_docs_arch["Architecture doc<br/>docs<br/>[ARCHITECTURE.md]"]
  node_n_docs_eval["Eval guide<br/>docs<br/>[EVAL_GUIDE.md]"]
end

node_n_runner -->|"calls"| node_n_inference
node_n_runner -->|"uses"| node_n_workflow
node_n_runner -->|"tracks"| node_n_steps
node_n_inference -->|"requests context"| node_n_ctx_mgr
node_n_inference -->|"validates"| node_n_guardrails
node_n_inference -->|"dispatches via"| node_n_client_base
node_n_inference -->|"shapes"| node_n_messages
node_n_ctx_mgr -->|"selects"| node_n_ctx_strat
node_n_ctx_mgr -->|"adapts to"| node_n_hardware
node_n_guardrails -->|"runs"| node_n_validator
node_n_guardrails -->|"enforces"| node_n_enforcer
node_n_guardrails -->|"repairs with"| node_n_nudge
node_n_prompts -->|"supplies"| node_n_nudge
node_n_client_base -->|"adapts"| node_n_client_ollama
node_n_client_base -->|"adapts"| node_n_client_llama
node_n_client_base -->|"adapts"| node_n_client_lf
node_n_client_base -->|"adapts"| node_n_client_anthropic
node_n_proxy -->|"translates"| node_n_proxy_convert
node_n_proxy -->|"routes"| node_n_proxy_handler
node_n_proxy_handler -->|"reuses"| node_n_inference
node_n_proxy_handler -->|"injects"| node_n_respond
node_n_respond -->|"finalized by"| node_n_guardrails
node_n_server -->|"launches backends"| node_n_client_base
node_n_server -->|"serves"| node_n_proxy
node_n_eval -->|"benchmarks"| node_n_runner
node_n_eval -->|"measures"| node_n_client_base
node_n_dashboard -->|"renders"| node_n_eval
node_n_docs_arch -.->|"documents"| node_n_runner
node_n_docs_eval -.->|"documents"| node_n_eval

click node_n_runner "https://github.com/antoinezambelli/forge/blob/main/src/forge/core/runner.py"
click node_n_inference "https://github.com/antoinezambelli/forge/blob/main/src/forge/core/inference.py"
click node_n_workflow "https://github.com/antoinezambelli/forge/blob/main/src/forge/core/workflow.py"
click node_n_messages "https://github.com/antoinezambelli/forge/blob/main/src/forge/core/messages.py"
click node_n_steps "https://github.com/antoinezambelli/forge/blob/main/src/forge/core/steps.py"
click node_n_slot_worker "https://github.com/antoinezambelli/forge/blob/main/src/forge/core/slot_worker.py"
click node_n_ctx_mgr "https://github.com/antoinezambelli/forge/blob/main/src/forge/context/manager.py"
click node_n_ctx_strat "https://github.com/antoinezambelli/forge/blob/main/src/forge/context/strategies.py"
click node_n_hardware "https://github.com/antoinezambelli/forge/blob/main/src/forge/context/hardware.py"
click node_n_guardrails "https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/guardrails.py"
click node_n_validator "https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/response_validator.py"
click node_n_enforcer "https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/step_enforcer.py"
click node_n_nudge "https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/nudge.py"
click node_n_client_base "https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py"
click node_n_client_ollama "https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py"
click node_n_client_llama "https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py"
click node_n_client_lf "https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/llamafile.py"
click node_n_client_anthropic "https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/anthropic.py"
click node_n_proxy "https://github.com/antoinezambelli/forge/blob/main/src/forge/proxy/server.py"
click node_n_proxy_convert "https://github.com/antoinezambelli/forge/blob/main/src/forge/proxy/convert.py"
click node_n_proxy_handler "https://github.com/antoinezambelli/forge/blob/main/src/forge/proxy/handler.py"
click node_n_server "https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py"
click node_n_prompts "https://github.com/antoinezambelli/forge/blob/main/src/forge/prompts/templates.py"
click node_n_respond "https://github.com/antoinezambelli/forge/blob/main/src/forge/tools/respond.py"
click node_n_eval "https://github.com/antoinezambelli/forge/blob/main/tests/eval/eval_runner.py"
click node_n_dashboard "https://github.com/antoinezambelli/forge/blob/main/tests/eval/dashboard/src/App.tsx"
click node_n_docs_arch "https://github.com/antoinezambelli/forge/blob/main/docs/ARCHITECTURE.md"
click node_n_docs_eval "https://github.com/antoinezambelli/forge/blob/main/docs/EVAL_GUIDE.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_runner,node_n_inference,node_n_workflow,node_n_messages,node_n_steps,node_n_slot_worker toneBlue
class node_n_client_base,node_n_client_ollama,node_n_client_llama,node_n_client_lf,node_n_client_anthropic,node_n_proxy,node_n_proxy_convert,node_n_proxy_handler,node_n_server,node_n_respond toneAmber
class node_n_ctx_mgr,node_n_ctx_strat,node_n_hardware,node_n_guardrails,node_n_validator,node_n_enforcer,node_n_nudge,node_n_prompts toneMint
class node_n_eval,node_n_dashboard toneRose
class node_n_docs_arch,node_n_docs_eval toneIndigo
```

## Components

### Core runtime

Workflow execution and shared inference path

- [WorkflowRunner](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/runner.py): orchestrator · End-to-end workflow execution and continuation loop · `src/forge/core/runner.py`
- [Inference](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/inference.py): pipeline · Shared front-half that prepares context and model requests · `src/forge/core/inference.py`
- [Workflow model](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/workflow.py): schema · Declarative workflow, tools, and terminal-step definitions · `src/forge/core/workflow.py`
- [Messages](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/messages.py): types · Core message and response representations · `src/forge/core/messages.py`
- [Steps](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/steps.py): state · Step tracking and progression rules · `src/forge/core/steps.py`
- [Slot worker](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/slot_worker.py): scheduler · Priority-queued shared inference access · `src/forge/core/slot_worker.py`

### I/O boundaries

Proxy and backend client interfaces

- [Client protocol](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py): adapter · Common LLM client boundary · `src/forge/clients/base.py`
- [Ollama](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py): backend adapter · Native function-calling client · `src/forge/clients/ollama.py`
- [Llama server](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py): backend adapter · llama.cpp or llama-server client modes · `src/forge/clients/base.py`
- [Llamafile](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/llamafile.py): backend adapter · Single-binary backend client · `src/forge/clients/llamafile.py`
- [Anthropic](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/anthropic.py): backend adapter · Hosted baseline client · `src/forge/clients/anthropic.py`
- [Proxy server](https://github.com/antoinezambelli/forge/blob/main/src/forge/proxy/server.py): http server · OpenAI-compatible asyncio proxy · `src/forge/proxy/server.py`
- [Convert](https://github.com/antoinezambelli/forge/blob/main/src/forge/proxy/convert.py): translation · OpenAI message translation layer · `src/forge/proxy/convert.py`
- [Handler](https://github.com/antoinezambelli/forge/blob/main/src/forge/proxy/handler.py): request flow · Proxy request execution and streaming · `src/forge/proxy/handler.py`
- [Server mgmt](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py): runtime · Backend startup and lifecycle coordination · `src/forge/server.py`
- [Respond tool](https://github.com/antoinezambelli/forge/blob/main/src/forge/tools/respond.py): synthetic tool · Synthetic tool used to coerce text output · `src/forge/tools/respond.py`

### Reliability

Context control and guardrails

- [Context manager](https://github.com/antoinezambelli/forge/blob/main/src/forge/context/manager.py): budgeting · Token-budget and compaction coordinator · `src/forge/context/manager.py`
- [Strategies](https://github.com/antoinezambelli/forge/blob/main/src/forge/context/strategies.py): compaction · Sliding-window and tiered context policies · `src/forge/context/strategies.py`
- [Hardware](https://github.com/antoinezambelli/forge/blob/main/src/forge/context/hardware.py): detection · Hardware-aware budget inputs · `src/forge/context/hardware.py`
- [Guardrails](https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/guardrails.py): recovery · Validation, retry, and failure recovery coordinator · `src/forge/guardrails/guardrails.py`
- [Validator](https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/response_validator.py): validation · Checks model responses and tool output shape · `src/forge/guardrails/response_validator.py`
- [Step enforcer](https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/step_enforcer.py): policy · Requires workflow step completion and order · `src/forge/guardrails/step_enforcer.py`
- [Nudges](https://github.com/antoinezambelli/forge/blob/main/src/forge/guardrails/nudge.py): repair · Retry prompts and corrective hints · `src/forge/guardrails/nudge.py`
- [Prompts](https://github.com/antoinezambelli/forge/blob/main/src/forge/prompts/templates.py): templates · System templates and retry prompt text · `src/forge/prompts/templates.py`

### Validation

Evaluation and regression harness

- [Eval suite](https://github.com/antoinezambelli/forge/blob/main/tests/eval/eval_runner.py): evaluation · Scenario-based regression and benchmark harness · `tests/eval/eval_runner.py`
- [Dashboard](https://github.com/antoinezambelli/forge/blob/main/tests/eval/dashboard/src/App.tsx): reporting ui · HTML/TSX results viewer for eval runs · `tests/eval/dashboard/src/App.tsx`

### Docs

Architecture and published guidance

- [Architecture doc](https://github.com/antoinezambelli/forge/blob/main/docs/ARCHITECTURE.md): docs · System architecture reference · `docs/ARCHITECTURE.md`
- [Eval guide](https://github.com/antoinezambelli/forge/blob/main/docs/EVAL_GUIDE.md): docs · Evaluation workflow and reporting guidance · `docs/EVAL_GUIDE.md`

## Connections

- WorkflowRunner → Inference: calls · Workflow execution begins with shared inference
- WorkflowRunner → Workflow model: uses · Workflow definitions drive execution
- WorkflowRunner → Steps: tracks · Advances and records workflow steps
- Inference → Context manager: requests context · Inference prepares token budget and compaction
- Inference → Guardrails: validates · Model output is checked before returning
- Inference → Client protocol: dispatches via · Inference calls the client abstraction
- Inference → Messages: shapes · Uses shared message and response types
- Context manager → Strategies: selects · Applies a compaction strategy
- Context manager → Hardware: adapts to · Budgeting reflects available hardware
- Guardrails → Validator: runs · Checks for malformed or incomplete responses
- Guardrails → Step enforcer: enforces · Ensures required workflow steps
- Guardrails → Nudges: repairs with · Retries are guided by corrective nudges
- Prompts → Nudges: supplies · Prompt templates back retry guidance
- Client protocol → Ollama: adapts · Concrete Ollama backend integration
- Client protocol → Llama server: adapts · llama-server / llama.cpp integration
- Client protocol → Llamafile: adapts · Llamafile integration
- Client protocol → Anthropic: adapts · Anthropic integration
- Proxy server → Convert: translates · Converts OpenAI-style payloads
- Proxy server → Handler: routes · Handles request execution and streaming
- Handler → Inference: reuses · Proxy shares the same inference path
- Handler → Respond tool: injects · Synthetic respond tool supports text coercion
- Respond tool → Guardrails: finalized by · Synthetic tool output is stripped or normalized
- Server mgmt → Client protocol: launches backends · Runtime management coordinates model servers
- Server mgmt → Proxy server: serves · Proxy-managed deployments use server lifecycle control
- Eval suite → WorkflowRunner: benchmarks · Evaluation exercises workflow execution
- Eval suite → Client protocol: measures · Eval runs against real backend clients
- Dashboard → Eval suite: renders · Dashboard visualizes evaluation outputs
- Architecture doc → WorkflowRunner: documents · Architecture doc describes the core runtime
- Eval guide → Eval suite: documents · Eval guide explains the harness and reporting
