# abhaychandra123/toolcalllm architecture

> Architecture diagram of the GitHub repository abhaychandra123/toolcalllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhaychandra123/toolcalllm
- Repository: https://github.com/abhaychandra123/toolcalllm

## Overview

1. **What this repo is**
   ToolCallLM is an end-to-end tool-calling fine-tuning pipeline centered on a base LLM (default Qwen3-8B). The core flow is: raw tool-call datasets in, standardized training examples out, LoRA/QLoRA fine-tuning, then inference, BFCL evaluation, and optional serving.

2. **Configuration is the main control plane**
   The project is driven by YAML recipes in `configs/`. `configs/base.yaml` defines shared defaults, and concrete configs extend it for training, inference, evaluation, FP8/Unsloth variants, BFCL subsets, and experiment-specific runs. `.env` supplies path/model overrides such as `DATA_DIR`, `OUTPUT_DIR`, and `BASE_MODEL`.

3. **Repository boundary: thin scripts, real logic in `src/`**
   `scripts/` contains command-line entrypoints only. They load config, call one function in `toolcall_lm.*`, and exit. The actual architecture lives under `src/toolcall_lm/`, which is split into small subsystems: `data`, `training`, `inference`, `api`, `eval`, `prompt`, `validators`, `tracking`, and `utils`.

4. **Data subsystem: normalize many dataset shapes into one schema**
   `src/toolcall_lm/data/` converts multiple source formats into a unified tool-calling schema. It supports OpenAI-chat style, ShareGPT, Alpaca-like records, BFCL-specific conversions, and dataset-specific adapters for ToolACE and xLAM. The flow is raw JSONL/JSON in `data/`, normalization and filtering in `preprocess.py`, then processed JSONL/manifests under `data/train/processed/`.

5. **Dataset quality gates happen early**
   The data pipeline includes schema validation, masking, truncation checks, and inspection utilities. Key boundaries are:
   - `schema.py` / `dataset.py` for the canonical record shape
   - `preprocess.py` for conversion and rejection logic
   - `masking.py`, `schema_jitter.py`, `thought_injection.py`, and `failure_driven_sampling.py` for augmentation/experimentation
   - `inspect.py` for human-readable validation before training
   This repo treats bad training rows as a first-class failure mode, not a downstream surprise.

6. **Prompting and output format are separate from the model**
   `src/toolcall_lm/prompt/` defines how tool specs and chat templates are rendered. The repo explicitly supports Qwen-style `<tool_call>...</tool_call>` outputs and fallback JSON formatting through `tool_format.py`. This boundary matters because training, inference, and validation all depend on the exact tool-call wrapper.

7. **Training subsystem: supervised fine-tuning with adapter-based methods**
   `src/toolcall_lm/training/` wires the model, tokenizer, LoRA/QLoRA config, callbacks, and watchdogs into the training loop. The architecture is adapter-first: base model remains fixed, while checkpoints store LoRA adapters plus tokenizer/config artifacts. There are distinct recipes for BF16 LoRA, QLoRA, DPO, FP8, Unsloth, and recovery-oriented experiments.

8. **Inference subsystem: same model, different boundary**
   `src/toolcall_lm/inference/` provides a CLI-facing engine that loads the base model plus an optional adapter and produces either raw text or parsed tool calls. It is also where validation of model outputs happens via `validators/json_parser.py` and `validators/tool_call_validator.py`. The inference layer is the boundary between free-form generation and structured tool-call semantics.

9. **Serving is a thin FastAPI wrapper over inference**
   `src/toolcall_lm/api/server.py` exposes health and chat-completions-style endpoints, with request/response schemas in `api/schemas.py`. The service reuses the same inference engine and prompt/tool formatting logic as the CLI, so the serving path is not a separate implementation. `scripts/serve.py` launches this API, and the README shows streaming support as an optional client-side concern.

10. **Evaluation is BFCL-centric and produces its own artifact tree**
   `src/toolcall_lm/eval/` contains the BFCL runner, Taubench runner, failure analysis, taxonomy, and run-comparison utilities. BFCL evaluation shells out to the official BFCL CLI, then aggregates category-level outputs into `outputs/bfcl_results/` and `outputs/bfcl_metrics/` style folders, including summaries and failure breakdowns. This is the main benchmark boundary for judging tool-calling quality.

11. **Synthetic and experimental dataset generation is a separate lane**
   The `synthetic_data_gen/` directory and several scripts under `scripts/` build synthetic multiturn, recovery, preference, and negative samples. These artifacts feed back into the same preprocessing pipeline, but they are clearly separated from model training and evaluation so the repo can support both curated and generated data sources.

12. **Operational helpers and infrastructure support**
   `src/toolcall_lm/utils/` contains configuration loading, logging, seeding, checkpoint utilities, environment sanitization, notifier hooks, and small runtime patches. The project also includes integrations for W&B tracking, optional Telegram/status notifications, Dockerfiles for different runtime setups, and `accelerate`/bitsandbytes/Unsloth paths where memory or performance constraints matter.

13. **Artifacts and state are intentionally kept out of source control**
   `data/` holds downloaded/raw/processed datasets, while `outputs/` is the workspace for checkpoints, merged models, BFCL results, metrics, logs, and generated reports. The many saved `merged_*` and `bfcl_*` folders in `outputs/` show that experiments are reproducible by config and artifact path rather than by hidden runtime state.

14. **Quality strategy: tests and diagnostic scripts**
   `tests/` validates the critical seams: preprocessing, masking, schema jitter, JSON parsing, LoRA config, toolace pipeline integrity, BFCL-related behavior, and end-to-end dry runs. The numerous scripts in `scripts/` act as targeted diagnostic tools for failure analysis, dataset audits, performance sweeps, and benchmark reproduction.

15. **How to diagram the system**
   A good architecture diagram should show one central pipeline with these stages: raw data sources → preprocessing/validation → processed training set → adapter fine-tuning → adapter checkpoint/merged model → inference/serving → BFCL/Taubench evaluation and analysis. Surround that pipeline with the config layer, prompt/validator boundary, and utility/ops layer that supports each stage.

Main entry points: [`src/toolcall_lm/training/trainer.py`](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/training/trainer.py) (Training), [`src/toolcall_lm/inference/engine.py`](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/inference/engine.py) (Inference engine), [`configs/base.yaml`](https://github.com/abhaychandra123/toolcalllm/blob/master/configs/base.yaml) (Base config), [`src/toolcall_lm/utils/config.py`](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/utils/config.py) (Utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_config["Config control plane"]
  node_configs_base["Base config<br/>yaml defaults<br/>[base.yaml]"]
  node_configs_train["Train recipes<br/>yaml recipes<br/>[train_common.yaml]"]
  node_configs_infer["Inference config<br/>yaml recipe<br/>[inference.yaml]"]
  node_configs_eval["Eval recipes<br/>yaml recipes<br/>[eval_bfcl.yaml]"]
end

subgraph group_group_data["Data pipeline"]
  node_data_raw[("Raw datasets<br/>source data")]
  node_data_schema["Canonical schema<br/>record schema<br/>[schema.py]"]
  node_data_preprocess["Preprocess<br/>conversion pipeline<br/>[preprocess.py]"]
  node_data_adapters["Dataset adapters<br/>source adapters<br/>[bfcl_format.py]"]
  node_data_quality["Quality gates<br/>validation and augmentation<br/>[inspect.py]"]
  node_data_processed[("Processed sets<br/>training artifacts")]
end

subgraph group_group_core["Core model flow"]
  node_prompting["Prompting<br/>rendering layer<br/>[templates.py]"]
  node_tool_format["Tool format<br/>output contract<br/>[tool_format.py]"]
  node_trainer["Training<br/>sft/adapters<br/>[trainer.py]"]
  node_dpo_unsloth["Alt trainers<br/>specialized training<br/>[dpo_trainer.py]"]
  node_checkpoint_artifacts[("Checkpoints<br/>adapter artifacts")]
  node_engine["Inference engine<br/>generation runtime<br/>[engine.py]"]
  node_validators["Validators<br/>output validation"]
  node_api_server["API server<br/>fastapi service<br/>[server.py]"]
end

subgraph group_group_eval["Evaluation"]
  node_bfcl_runner["BFCL runner<br/>benchmark harness<br/>[bfcl_runner.py]"]
  node_taubench_runner["Taubench<br/>benchmark harness<br/>[taubench_runner.py]"]
  node_analysis["Failure analysis<br/>metrics and taxonomy"]
end

subgraph group_group_ops["Ops and synthetic"]
  node_utils["Utilities<br/>runtime support<br/>[config.py]"]
  node_tracking["Tracking<br/>wandb setup<br/>[wandb_setup.py]"]
  node_synthetic_gen["Synthetic gen<br/>data generation"]
end

node_configs_base -->|"extends"| node_configs_train
node_configs_base -->|"extends"| node_configs_infer
node_configs_base -->|"extends"| node_configs_eval
node_configs_train -->|"drives"| node_trainer
node_configs_infer -->|"drives"| node_engine
node_configs_eval -->|"drives"| node_bfcl_runner
node_data_raw -->|"ingested by"| node_data_adapters
node_data_adapters -->|"maps to"| node_data_schema
node_data_schema -->|"validated by"| node_data_preprocess
node_data_preprocess -->|"gated by"| node_data_quality
node_data_quality -->|"produces"| node_data_processed
node_synthetic_gen -->|"feeds"| node_data_preprocess
node_prompting -->|"renders"| node_tool_format
node_tool_format -->|"constrains"| node_engine
node_prompting -->|"uses"| node_trainer
node_data_processed -->|"trains"| node_trainer
node_trainer -->|"writes"| node_checkpoint_artifacts
node_checkpoint_artifacts -->|"loads"| node_engine
node_engine -->|"checks"| node_validators
node_engine -->|"serves"| node_api_server
node_bfcl_runner -->|"summarizes"| node_analysis
node_taubench_runner -->|"summarizes"| node_analysis
node_trainer -->|"reports"| node_tracking
node_utils -->|"supports"| node_trainer
node_utils -->|"supports"| node_engine
node_utils -->|"supports"| node_api_server

click node_configs_base "https://github.com/abhaychandra123/toolcalllm/blob/master/configs/base.yaml"
click node_configs_train "https://github.com/abhaychandra123/toolcalllm/blob/master/configs/train_common.yaml"
click node_configs_infer "https://github.com/abhaychandra123/toolcalllm/blob/master/configs/inference.yaml"
click node_configs_eval "https://github.com/abhaychandra123/toolcalllm/blob/master/configs/eval_bfcl.yaml"
click node_data_raw "https://github.com/abhaychandra123/toolcalllm/tree/master/data"
click node_data_schema "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/schema.py"
click node_data_preprocess "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/preprocess.py"
click node_data_adapters "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/bfcl_format.py"
click node_data_quality "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/inspect.py"
click node_data_processed "https://github.com/abhaychandra123/toolcalllm/tree/master/data/train/processed"
click node_prompting "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/prompt/templates.py"
click node_tool_format "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/prompt/tool_format.py"
click node_trainer "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/training/trainer.py"
click node_dpo_unsloth "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/training/dpo_trainer.py"
click node_checkpoint_artifacts "https://github.com/abhaychandra123/toolcalllm/tree/master/outputs/merged_qwen3_blended_exp2"
click node_engine "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/inference/engine.py"
click node_validators "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/validators/tool_call_validator.py"
click node_api_server "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/api/server.py"
click node_bfcl_runner "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/eval/bfcl_runner.py"
click node_taubench_runner "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/eval/taubench_runner.py"
click node_analysis "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/eval/failure_analysis.py"
click node_utils "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/utils/config.py"
click node_tracking "https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/tracking/wandb_setup.py"
click node_synthetic_gen "https://github.com/abhaychandra123/toolcalllm/blob/master/synthetic_data_gen/generate_synthetic_multiturn.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_configs_base,node_configs_train,node_configs_infer,node_configs_eval toneBlue
class node_data_raw,node_data_schema,node_data_preprocess,node_data_adapters,node_data_quality,node_data_processed toneAmber
class node_prompting,node_tool_format,node_trainer,node_dpo_unsloth,node_checkpoint_artifacts,node_engine,node_validators,node_api_server toneMint
class node_bfcl_runner,node_taubench_runner,node_analysis toneRose
class node_utils,node_tracking,node_synthetic_gen toneIndigo
```

## Components

### Config control plane

YAML recipes and environment overrides that steer the pipeline.

- [Base config](https://github.com/abhaychandra123/toolcalllm/blob/master/configs/base.yaml): yaml defaults · Shared YAML defaults for model, paths, and runtime behavior. · `configs/base.yaml`
- [Train recipes](https://github.com/abhaychandra123/toolcalllm/blob/master/configs/train_common.yaml): yaml recipes · Training variants for LoRA, QLoRA, DPO, FP8, Unsloth, and experiments. · `configs/train_common.yaml`
- [Inference config](https://github.com/abhaychandra123/toolcalllm/blob/master/configs/inference.yaml): yaml recipe · CLI inference and model-loading settings. · `configs/inference.yaml`
- [Eval recipes](https://github.com/abhaychandra123/toolcalllm/blob/master/configs/eval_bfcl.yaml): yaml recipes · BFCL and Taubench evaluation configurations. · `configs/eval_bfcl.yaml`

### Data pipeline

Raw datasets, normalization, validation, and processed training artifacts.

- [Raw datasets](https://github.com/abhaychandra123/toolcalllm/tree/master/data): source data · Downloaded JSON/JSONL datasets such as ToolACE, xLAM, BFCL, and custom blends. · `data`
- [Canonical schema](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/schema.py): record schema · Unified tool-calling record shape used across sources. · `src/toolcall_lm/data/schema.py`
- [Preprocess](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/preprocess.py): conversion pipeline · Converts, filters, truncates, and rejects malformed examples. · `src/toolcall_lm/data/preprocess.py`
- [Dataset adapters](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/bfcl_format.py): source adapters · Format-specific handlers for BFCL, ToolACE, xLAM, and blend validation. · `src/toolcall_lm/data/bfcl_format.py`
- [Quality gates](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/data/inspect.py): validation and augmentation · Masking, schema jitter, thought injection, failure-driven sampling, and inspection. · `src/toolcall_lm/data/inspect.py`
- [Processed sets](https://github.com/abhaychandra123/toolcalllm/tree/master/data/train/processed): training artifacts · Normalized JSONL manifests and derived training splits. · `data/train/processed`

### Core model flow

Training, inference, prompts, validation, and API serving.

- [Prompting](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/prompt/templates.py): rendering layer · Renders tool specs, chat templates, and tool-call wrappers. · `src/toolcall_lm/prompt/templates.py`
- [Tool format](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/prompt/tool_format.py): output contract · Qwen-style tool-call wrapper and JSON fallback formatting. · `src/toolcall_lm/prompt/tool_format.py`
- [Training](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/training/trainer.py): sft/adapters · Wires model, tokenizer, LoRA/QLoRA, callbacks, and watchdogs. · `src/toolcall_lm/training/trainer.py`
- [Alt trainers](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/training/dpo_trainer.py): specialized training · DPO and Unsloth-based training paths for specific recipes. · `src/toolcall_lm/training/dpo_trainer.py`
- [Checkpoints](https://github.com/abhaychandra123/toolcalllm/tree/master/outputs/merged_qwen3_blended_exp2): adapter artifacts · Adapter checkpoints, tokenizer files, merged models, and config bundles. · `outputs/merged_qwen3_blended_exp2`
- [Inference engine](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/inference/engine.py): generation runtime · Loads base model plus optional adapter and produces text or tool calls. · `src/toolcall_lm/inference/engine.py`
- [Validators](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/validators/tool_call_validator.py): output validation · Parses JSON and checks tool-call structure during inference. · `src/toolcall_lm/validators/tool_call_validator.py`
- [API server](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/api/server.py): fastapi service · Thin serving layer exposing health and chat-completions-style endpoints. · `src/toolcall_lm/api/server.py`

### Evaluation

Benchmark runners and analysis for tool-calling quality.

- [BFCL runner](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/eval/bfcl_runner.py): benchmark harness · Executes BFCL evaluation and collects per-task results. · `src/toolcall_lm/eval/bfcl_runner.py`
- [Taubench](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/eval/taubench_runner.py): benchmark harness · Runs the Taubench evaluation path. · `src/toolcall_lm/eval/taubench_runner.py`
- [Failure analysis](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/eval/failure_analysis.py): metrics and taxonomy · Comparisons, taxonomy, event logs, and error breakdowns. · `src/toolcall_lm/eval/failure_analysis.py`

### Ops and synthetic

Supporting utilities, tracking, and generated datasets.

- [Utilities](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/utils/config.py): runtime support · Config loading, logging, seeding, checkpoint helpers, env sanitization, and patches. · `src/toolcall_lm/utils/config.py`
- [Tracking](https://github.com/abhaychandra123/toolcalllm/blob/master/src/toolcall_lm/tracking/wandb_setup.py): wandb setup · W&B initialization and experiment logging integration. · `src/toolcall_lm/tracking/wandb_setup.py`
- [Synthetic gen](https://github.com/abhaychandra123/toolcalllm/blob/master/synthetic_data_gen/generate_synthetic_multiturn.py): data generation · Generates multiturn, recovery, preference, and negative samples. · `synthetic_data_gen/generate_synthetic_multiturn.py`

## Connections

- Base config → Train recipes: extends · Training recipes inherit shared defaults.
- Base config → Inference config: extends · Inference settings reuse base paths and model defaults.
- Base config → Eval recipes: extends · Evaluation recipes share the same control plane.
- Train recipes → Training: drives · Training recipes configure the adapter-based trainer.
- Inference config → Inference engine: drives · Inference recipes configure model loading and generation.
- Eval recipes → BFCL runner: drives · BFCL configs parameterize benchmark execution.
- Raw datasets → Dataset adapters: ingested by · Raw dataset shapes are normalized through adapters.
- Dataset adapters → Canonical schema: maps to · Adapters emit the canonical record schema.
- Canonical schema → Preprocess: validated by · Preprocess enforces the canonical shape and rejection rules.
- Preprocess → Quality gates: gated by · Examples are filtered and inspected through quality checks.
- Quality gates → Processed sets: produces · Validated examples become processed manifests and splits.
- Synthetic gen → Preprocess: feeds · Synthetic datasets re-enter the same preprocessing pipeline.
- Prompting → Tool format: renders · Prompt templates emit the required tool-call wrapper.
- Tool format → Inference engine: constrains · Inference must honor the same structured output contract.
- Prompting → Training: uses · Training and inference share prompt construction.
- Processed sets → Training: trains · Processed datasets feed supervised and adapter-based training.
- Training → Checkpoints: writes · Training produces adapter checkpoints and merged artifacts.
- Checkpoints → Inference engine: loads · Inference can load a base model plus adapter artifacts.
- Inference engine → Validators: checks · Generated text is parsed and validated as tool calls.
- Inference engine → API server: serves · The API wrapper reuses the same inference engine.
- BFCL runner → Failure analysis: summarizes · BFCL outputs are aggregated into failure and taxonomy views.
- Taubench → Failure analysis: summarizes · Taubench runs feed the same analysis layer.
- Training → Tracking: reports · Training can log runs and metrics to W&B.
- Utilities → Training: supports · Trainer relies on shared runtime helpers.
- Utilities → Inference engine: supports · Inference uses common config, logging, and checkpoint helpers.
- Utilities → API server: supports · Serving uses shared runtime utilities and status helpers.