How toolbrain/toolbrain works
1. Core shape of the repo
ToolBrain is a Python 3.10+ library, not an application. The architecture centers on a Brain orchestrator that trains agentic systems by wrapping an existing agent framework, collecting execution traces, scoring them with rewards, and updating the policy with one of several learning algorithms.
2. Main runtime boundary: agent framework adapters
The library does not own the agent runtime directly. Instead, it integrates with external agent frameworks through adapters under toolbrain/adapters/. Two concrete integrations are present: SmolAgents and LangChain. These adapters translate framework-specific agent state and tool calls into ToolBrain’s internal trace format.
3. Orchestration layer
toolbrain/brain.py is the top-level entry point used by examples and users. It likely coordinates the full loop: initialize adapter, run the agent on training samples, collect traces, compute rewards, and dispatch training updates to the selected algorithm. toolbrain/factory.py suggests a construction layer for assembling the right adapter/algorithm combination from configuration.
4. Internal shared data model
toolbrain/core_types.py is the repository’s boundary for shared structured objects. This is important for architecture because it implies the system standardizes on internal types for traces, prompts, policy outputs, and possibly tool definitions, so adapters and learning algorithms can remain decoupled from framework-specific objects.
5. Learning subsystem
The toolbrain/learning/ package contains the optimization layer. It is split into three modes: supervised learning, DPO, and GRPO. Each mode has its own algo.py, and GRPO additionally has losses.py, indicating a separate objective implementation. policy.py and the utils.py files suggest common abstractions around sampling, rollout handling, or policy updates.
6. Reward subsystem
toolbrain/rewards.py defines reward functions and likely the default reward utilities used by training examples. The README emphasizes customizable Python reward functions and LLM-as-judge workflows, so reward computation is a first-class boundary between trace collection and learning.
7. Retrieval and tool selection subsystem
toolbrain/retriever.py and the example examples/06_tool_retrieval.py indicate a tool-management layer for large tool inventories. This subsystem sits between the agent and training loop to choose or narrow tool sets before execution, which is important for scaling beyond small toy tool collections.
8. Prompting and model-facing abstractions
toolbrain/prompt.py and toolbrain/models.py suggest shared abstractions for prompt construction and model representation. These are likely used both by adapters and training algorithms to produce consistent training inputs across different backends.
9. Example workflows as architectural guides
The examples/ directory is a map of supported end-to-end flows: GRPO, DPO, supervised training, zero-learn task generation, distillation, flexible rewards, LangChain and SmolAgents integration, HPO, and resource-efficient fine-tuning variants such as FP16 and BitsAndBytes. The examples show that the library is designed as a training framework around existing agent stacks, not as a standalone agent runtime.
10. Documentation and research artifacts
docs/ is a full Sphinx documentation site with tutorials and API modules, while data/ contains at least one generated learning curve artifact used in the README. This suggests the repo is intended to support both library usage and reproducible research/demo workflows.
11. Typical data flow through the system
A user creates an external agent with tools, passes it into Brain, supplies a dataset of training tasks, and selects an algorithm. ToolBrain then runs episodes through the adapter, records execution traces, applies reward functions, and feeds the resulting signals into supervised/DPO/GRPO updates. The output is improved agent behavior, usually with better tool selection or tool-use reliability.
12. Architectural boundaries to draw
For an architecture diagram, the key boundaries are: user code/examples at the top; adapter layer to external agent frameworks; trace/reward interface in the middle; learning algorithms as the optimization core; and supporting utilities for config, prompting, retrieval, and model abstractions around that core. The strongest external dependencies implied by the repo are SmolAgents, LangChain, and common Python ML/LLM tooling used by the examples.
Main entry points: brain.py Brain · policy.py Policy · learning Learning utils · examples Examples
Components 20
User & examples
Entry points, sample workflows, and documentation that show how the library is used.
- Examples · End-to-end training and integration demos for different agent stacks and learning modes.
examples
Runtime boundary
Orchestration and adapters that connect external agent frameworks to ToolBrain's internal traces.
- Brain · Top-level training loop entry point that coordinates adapters, traces, rewards, and updates.
toolbrain/brain.py - Factory · Builds the right adapter and learning setup from configuration.
toolbrain/factory.py - Adapter base · Common adapter contract for wrapping external agent frameworks.
toolbrain/adapters/base_adapter.py - SmolAgents · Integration for SmolAgents agents and tool calls.
toolbrain/adapters/smolagent/smolagent_adapter.py - LangChain · Integration for LangChain agents and tools.
toolbrain/adapters/langchain/langchain_adapter.py - HF tool wrap · LangChain-facing wrapper for hosted or external tools.
toolbrain/adapters/langchain/hf_tool_wrapper.py
Optimization core
Shared types, reward shaping, retrieval, prompting, and learning algorithms.
- Core types · Internal trace, prompt, policy, and tool-related data structures.
toolbrain/core_types.py - Rewards · Reward computation boundary, including custom and judge-based rewards.
toolbrain/rewards.py - Retriever · Narrows or selects tools for large tool inventories.
toolbrain/retriever.py - Prompting · Constructs model-facing prompts and training inputs.
toolbrain/prompt.py - Models · Shared model-facing representations used by adapters and training code.
toolbrain/models.py - Policy · Common policy interface used by the learning algorithms.
toolbrain/learning/policy.py - Supervised · Supervised training path for imitation-style updates.
toolbrain/learning/supervised/algo.py - DPO · Direct preference optimization training path.
toolbrain/learning/dpo/algo.py - GRPO · Group-relative policy optimization training path.
toolbrain/learning/grpo/algo.py - GRPO losses · Loss functions specific to the GRPO optimizer.
toolbrain/learning/grpo/losses.py - Learning utils · Shared utilities for training workflows across optimization modes.
toolbrain/learning
Connections 23
- Examples → Brain: use · Examples invoke the main training orchestrator.
- Docs site → Examples: document · Tutorials and API docs explain the example workflows.
- Brain → Factory: builds via · Brain relies on factory composition for configuration-driven setup.
- Factory → Adapter base: selects · Factory assembles the appropriate adapter implementation.
- Adapter base → SmolAgents: implements · SmolAgents adapter follows the common adapter contract.
- Adapter base → LangChain: implements · LangChain adapter follows the common adapter contract.
- LangChain → HF tool wrap: uses · LangChain integration includes a helper for wrapping tools.
- Brain → Core types: exchanges · Training loop moves traces and structured objects through shared types.
- SmolAgents → Core types: maps to · Framework state and tool calls are converted into internal trace types.
- LangChain → Core types: maps to · Framework state and tool calls are converted into internal trace types.
- Brain → Rewards: scores with · Collected traces are evaluated by reward functions.
- Brain → Retriever: optionally narrows · Tool selection can be applied before agent execution.
- Brain → Prompting: formats · Prompt construction feeds the agent and training inputs.
- Prompting → Models: targets · Prompts and outputs are aligned with shared model abstractions.
- Brain → Policy: updates · The orchestrator dispatches learning signals into the active policy.
- Policy → Supervised: used by · Supervised mode uses the shared policy abstraction.
- Policy → DPO: used by · DPO mode uses the shared policy abstraction.
- Policy → GRPO: used by · GRPO mode uses the shared policy abstraction.
- GRPO → GRPO losses: depends on · GRPO uses specialized objective functions.
- Learning utils → Supervised: supports · Shared helpers support supervised training flows.
- Learning utils → DPO: supports · Shared helpers support DPO training flows.
- Learning utils → GRPO: supports · Shared helpers support GRPO training flows.
- Examples → Artifacts: produces · Some example workflows generate plots and experiment artifacts.