# toolbrain/toolbrain architecture

> Architecture diagram of the GitHub repository toolbrain/toolbrain, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/toolbrain/toolbrain
- Repository: https://github.com/toolbrain/toolbrain

## Overview

1. Core shape of the repo  
ToolBrain is a Python 3.10+ library, not an application. The architecture centers on a `Brain` orchestrator that trains agentic systems by wrapping an existing agent framework, collecting execution traces, scoring them with rewards, and updating the policy with one of several learning algorithms.

2. Main runtime boundary: agent framework adapters  
The library does not own the agent runtime directly. Instead, it integrates with external agent frameworks through adapters under `toolbrain/adapters/`. Two concrete integrations are present: SmolAgents and LangChain. These adapters translate framework-specific agent state and tool calls into ToolBrain’s internal trace format.

3. Orchestration layer  
`toolbrain/brain.py` is the top-level entry point used by examples and users. It likely coordinates the full loop: initialize adapter, run the agent on training samples, collect traces, compute rewards, and dispatch training updates to the selected algorithm. `toolbrain/factory.py` suggests a construction layer for assembling the right adapter/algorithm combination from configuration.

4. Internal shared data model  
`toolbrain/core_types.py` is the repository’s boundary for shared structured objects. This is important for architecture because it implies the system standardizes on internal types for traces, prompts, policy outputs, and possibly tool definitions, so adapters and learning algorithms can remain decoupled from framework-specific objects.

5. Learning subsystem  
The `toolbrain/learning/` package contains the optimization layer. It is split into three modes: supervised learning, DPO, and GRPO. Each mode has its own `algo.py`, and GRPO additionally has `losses.py`, indicating a separate objective implementation. `policy.py` and the `utils.py` files suggest common abstractions around sampling, rollout handling, or policy updates.

6. Reward subsystem  
`toolbrain/rewards.py` defines reward functions and likely the default reward utilities used by training examples. The README emphasizes customizable Python reward functions and LLM-as-judge workflows, so reward computation is a first-class boundary between trace collection and learning.

7. Retrieval and tool selection subsystem  
`toolbrain/retriever.py` and the example `examples/06_tool_retrieval.py` indicate a tool-management layer for large tool inventories. This subsystem sits between the agent and training loop to choose or narrow tool sets before execution, which is important for scaling beyond small toy tool collections.

8. Prompting and model-facing abstractions  
`toolbrain/prompt.py` and `toolbrain/models.py` suggest shared abstractions for prompt construction and model representation. These are likely used both by adapters and training algorithms to produce consistent training inputs across different backends.

9. Example workflows as architectural guides  
The `examples/` directory is a map of supported end-to-end flows: GRPO, DPO, supervised training, zero-learn task generation, distillation, flexible rewards, LangChain and SmolAgents integration, HPO, and resource-efficient fine-tuning variants such as FP16 and BitsAndBytes. The examples show that the library is designed as a training framework around existing agent stacks, not as a standalone agent runtime.

10. Documentation and research artifacts  
`docs/` is a full Sphinx documentation site with tutorials and API modules, while `data/` contains at least one generated learning curve artifact used in the README. This suggests the repo is intended to support both library usage and reproducible research/demo workflows.

11. Typical data flow through the system  
A user creates an external agent with tools, passes it into `Brain`, supplies a dataset of training tasks, and selects an algorithm. ToolBrain then runs episodes through the adapter, records execution traces, applies reward functions, and feeds the resulting signals into supervised/DPO/GRPO updates. The output is improved agent behavior, usually with better tool selection or tool-use reliability.

12. Architectural boundaries to draw  
For an architecture diagram, the key boundaries are: user code/examples at the top; adapter layer to external agent frameworks; trace/reward interface in the middle; learning algorithms as the optimization core; and supporting utilities for config, prompting, retrieval, and model abstractions around that core. The strongest external dependencies implied by the repo are SmolAgents, LangChain, and common Python ML/LLM tooling used by the examples.

Main entry points: [`toolbrain/brain.py`](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/brain.py) (Brain), [`toolbrain/learning/policy.py`](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/policy.py) (Policy), [`toolbrain/learning`](https://github.com/toolbrain/toolbrain/tree/main/toolbrain/learning) (Learning utils), [`examples`](https://github.com/toolbrain/toolbrain/tree/main/examples) (Examples)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_user["User & examples"]
  node_node_examples["Examples<br/>workflows"]
end

subgraph group_group_runtime["Runtime boundary"]
  node_node_brain["Brain<br/>orchestrator<br/>[brain.py]"]
  node_node_factory["Factory<br/>composition<br/>[factory.py]"]
  node_node_base_adapter["Adapter base<br/>adapter interface<br/>[base_adapter.py]"]
  node_node_smolagent["SmolAgents<br/>adapter"]
  node_node_langchain["LangChain<br/>adapter"]
  node_node_hf_tool_wrapper["HF tool wrap<br/>integration helper<br/>[hf_tool_wrapper.py]"]
end

subgraph group_group_learning["Optimization core"]
  node_node_core_types["Core types<br/>shared model<br/>[core_types.py]"]
  node_node_rewards["Rewards<br/>reward functions<br/>[rewards.py]"]
  node_node_retriever["Retriever<br/>tool selection<br/>[retriever.py]"]
  node_node_prompt["Prompting<br/>prompt builder<br/>[prompt.py]"]
  node_node_models["Models<br/>model abstractions<br/>[models.py]"]
  node_node_policy["Policy<br/>[policy.py]"]
  node_node_supervised["Supervised<br/>learning algo<br/>[algo.py]"]
  node_node_dpo["DPO<br/>learning algo<br/>[algo.py]"]
  node_node_grpo["GRPO<br/>learning algo<br/>[algo.py]"]
  node_node_grpo_losses["GRPO losses<br/>objective<br/>[losses.py]"]
  node_node_learning_utils["Learning utils<br/>helpers"]
end

subgraph group_group_docs["Docs & artifacts"]
  node_node_docs["Docs site<br/>sphinx docs"]
  node_node_data["Artifacts<br/>demo outputs"]
end

node_node_examples -->|"use"| node_node_brain
node_node_docs -->|"document"| node_node_examples
node_node_brain -->|"builds via"| node_node_factory
node_node_factory -->|"selects"| node_node_base_adapter
node_node_base_adapter -->|"implements"| node_node_smolagent
node_node_base_adapter -->|"implements"| node_node_langchain
node_node_langchain -->|"uses"| node_node_hf_tool_wrapper
node_node_brain -->|"exchanges"| node_node_core_types
node_node_smolagent -->|"maps to"| node_node_core_types
node_node_langchain -->|"maps to"| node_node_core_types
node_node_brain -->|"scores with"| node_node_rewards
node_node_brain -.->|"optionally narrows"| node_node_retriever
node_node_brain -->|"formats"| node_node_prompt
node_node_prompt -->|"targets"| node_node_models
node_node_brain -->|"updates"| node_node_policy
node_node_policy -->|"used by"| node_node_supervised
node_node_policy -->|"used by"| node_node_dpo
node_node_policy -->|"used by"| node_node_grpo
node_node_grpo -->|"depends on"| node_node_grpo_losses
node_node_learning_utils -.->|"supports"| node_node_supervised
node_node_learning_utils -.->|"supports"| node_node_dpo
node_node_learning_utils -.->|"supports"| node_node_grpo
node_node_examples -.->|"produces"| node_node_data

click node_node_examples "https://github.com/toolbrain/toolbrain/tree/main/examples"
click node_node_docs "https://github.com/toolbrain/toolbrain/tree/main/docs"
click node_node_data "https://github.com/toolbrain/toolbrain/tree/main/data"
click node_node_brain "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/brain.py"
click node_node_factory "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/factory.py"
click node_node_base_adapter "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/base_adapter.py"
click node_node_smolagent "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/smolagent/smolagent_adapter.py"
click node_node_langchain "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/langchain/langchain_adapter.py"
click node_node_hf_tool_wrapper "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/langchain/hf_tool_wrapper.py"
click node_node_core_types "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/core_types.py"
click node_node_rewards "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/rewards.py"
click node_node_retriever "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/retriever.py"
click node_node_prompt "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/prompt.py"
click node_node_models "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/models.py"
click node_node_policy "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/policy.py"
click node_node_supervised "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/supervised/algo.py"
click node_node_dpo "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/dpo/algo.py"
click node_node_grpo "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/grpo/algo.py"
click node_node_grpo_losses "https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/grpo/losses.py"
click node_node_learning_utils "https://github.com/toolbrain/toolbrain/tree/main/toolbrain/learning"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_examples toneBlue
class node_node_brain,node_node_factory,node_node_base_adapter,node_node_smolagent,node_node_langchain,node_node_hf_tool_wrapper toneAmber
class node_node_core_types,node_node_rewards,node_node_retriever,node_node_prompt,node_node_models,node_node_policy,node_node_supervised,node_node_dpo,node_node_grpo,node_node_grpo_losses,node_node_learning_utils toneMint
class node_node_docs,node_node_data toneRose
```

## Components

### User & examples

Entry points, sample workflows, and documentation that show how the library is used.

- [Examples](https://github.com/toolbrain/toolbrain/tree/main/examples): workflows · End-to-end training and integration demos for different agent stacks and learning modes. · `examples`

### Runtime boundary

Orchestration and adapters that connect external agent frameworks to ToolBrain's internal traces.

- [Brain](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/brain.py): orchestrator · Top-level training loop entry point that coordinates adapters, traces, rewards, and updates. · `toolbrain/brain.py`
- [Factory](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/factory.py): composition · Builds the right adapter and learning setup from configuration. · `toolbrain/factory.py`
- [Adapter base](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/base_adapter.py): adapter interface · Common adapter contract for wrapping external agent frameworks. · `toolbrain/adapters/base_adapter.py`
- [SmolAgents](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/smolagent/smolagent_adapter.py): adapter · Integration for SmolAgents agents and tool calls. · `toolbrain/adapters/smolagent/smolagent_adapter.py`
- [LangChain](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/langchain/langchain_adapter.py): adapter · Integration for LangChain agents and tools. · `toolbrain/adapters/langchain/langchain_adapter.py`
- [HF tool wrap](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/adapters/langchain/hf_tool_wrapper.py): integration helper · LangChain-facing wrapper for hosted or external tools. · `toolbrain/adapters/langchain/hf_tool_wrapper.py`

### Optimization core

Shared types, reward shaping, retrieval, prompting, and learning algorithms.

- [Core types](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/core_types.py): shared model · Internal trace, prompt, policy, and tool-related data structures. · `toolbrain/core_types.py`
- [Rewards](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/rewards.py): reward functions · Reward computation boundary, including custom and judge-based rewards. · `toolbrain/rewards.py`
- [Retriever](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/retriever.py): tool selection · Narrows or selects tools for large tool inventories. · `toolbrain/retriever.py`
- [Prompting](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/prompt.py): prompt builder · Constructs model-facing prompts and training inputs. · `toolbrain/prompt.py`
- [Models](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/models.py): model abstractions · Shared model-facing representations used by adapters and training code. · `toolbrain/models.py`
- [Policy](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/policy.py): policy abstraction · Common policy interface used by the learning algorithms. · `toolbrain/learning/policy.py`
- [Supervised](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/supervised/algo.py): learning algo · Supervised training path for imitation-style updates. · `toolbrain/learning/supervised/algo.py`
- [DPO](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/dpo/algo.py): learning algo · Direct preference optimization training path. · `toolbrain/learning/dpo/algo.py`
- [GRPO](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/grpo/algo.py): learning algo · Group-relative policy optimization training path. · `toolbrain/learning/grpo/algo.py`
- [GRPO losses](https://github.com/toolbrain/toolbrain/blob/main/toolbrain/learning/grpo/losses.py): objective · Loss functions specific to the GRPO optimizer. · `toolbrain/learning/grpo/losses.py`
- [Learning utils](https://github.com/toolbrain/toolbrain/tree/main/toolbrain/learning): helpers · Shared utilities for training workflows across optimization modes. · `toolbrain/learning`

### Docs & artifacts

Documentation site and generated outputs used for tutorials and demos.

- [Docs site](https://github.com/toolbrain/toolbrain/tree/main/docs): sphinx docs · Tutorials and API documentation for using the library. · `docs`
- [Artifacts](https://github.com/toolbrain/toolbrain/tree/main/data): demo outputs · Generated learning-curve or experiment artifacts referenced by the examples and README. · `data`

## Connections

- Examples → Brain: use · Examples invoke the main training orchestrator.
- Docs site → Examples: document · Tutorials and API docs explain the example workflows.
- Brain → Factory: builds via · Brain relies on factory composition for configuration-driven setup.
- Factory → Adapter base: selects · Factory assembles the appropriate adapter implementation.
- Adapter base → SmolAgents: implements · SmolAgents adapter follows the common adapter contract.
- Adapter base → LangChain: implements · LangChain adapter follows the common adapter contract.
- LangChain → HF tool wrap: uses · LangChain integration includes a helper for wrapping tools.
- Brain → Core types: exchanges · Training loop moves traces and structured objects through shared types.
- SmolAgents → Core types: maps to · Framework state and tool calls are converted into internal trace types.
- LangChain → Core types: maps to · Framework state and tool calls are converted into internal trace types.
- Brain → Rewards: scores with · Collected traces are evaluated by reward functions.
- Brain → Retriever: optionally narrows · Tool selection can be applied before agent execution.
- Brain → Prompting: formats · Prompt construction feeds the agent and training inputs.
- Prompting → Models: targets · Prompts and outputs are aligned with shared model abstractions.
- Brain → Policy: updates · The orchestrator dispatches learning signals into the active policy.
- Policy → Supervised: used by · Supervised mode uses the shared policy abstraction.
- Policy → DPO: used by · DPO mode uses the shared policy abstraction.
- Policy → GRPO: used by · GRPO mode uses the shared policy abstraction.
- GRPO → GRPO losses: depends on · GRPO uses specialized objective functions.
- Learning utils → Supervised: supports · Shared helpers support supervised training flows.
- Learning utils → DPO: supports · Shared helpers support DPO training flows.
- Learning utils → GRPO: supports · Shared helpers support GRPO training flows.
- Examples → Artifacts: produces · Some example workflows generate plots and experiment artifacts.
