# hinthornw/trustcall architecture

> Architecture diagram of the GitHub repository hinthornw/trustcall, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hinthornw/trustcall
- Repository: https://github.com/hinthornw/trustcall

## Overview

##### Project Type and Purpose
- This is a Python library/package (`trustcall`) designed to improve reliability and efficiency when LLMs generate or update structured JSON outputs.
- It integrates with LLMs (OpenAI, LangChain‐Fireworks, etc.) and pydantic (or user‐defined Python functions) to iteratively generate JSON patches rather than full documents, handling validation errors gracefully.

##### File Structure Overview
- Top‐level:
- `.github/workflows/` – CI pipelines for tests and evals.
- `pyproject.toml` – packaging and dependencies.
- `Makefile` – shortcuts (`make test`, `make evals`).
- `_static/` – images used in README.
- Core code under `trustcall/`:
- `_base.py` – core extractor logic (prompt LLM, apply patches, retry).
- `_validation_node.py` – LangChain integration node for tool calling.
- `py.typed` – typed package.
- Tests under `tests/`:
- Unit tests (`tests/unit_tests/`) for utils, extraction, strict validation.
- Eval tests (`tests/evals/`) with VCR cassettes, benchmarking LLM performance.

##### Key Components
a. **User Application** – Your code that calls `create_extractor(...)` or `llm.with_structured_output(...)`.
b. **Trustcall Library** –
- **Extractor Core** (`_base.py`): orchestrates prompts, applies patches, handles retries.
- **Validation Node** (`_validation_node.py`): wraps LangChain tool‐calling interface.
c. **LLM Providers** – OpenAI, ChatFireworks, or any LangChain‐compatible model endpoint.
d. **Schema Validators** – pydantic models or user functions defining JSON schema.
e. **JSON Patch Engine** – RFC6902 patch creation and application.
f. **Test Harness & CI** – pytest suite, VCR cassette playback, GitHub Actions workflows.

##### Relationships and Interactions
- **User App → Trustcall Extractor**: code invokes the extractor with schema/tool definitions and (optionally) existing JSON.
- **Extractor → LLM Endpoint**: sends prompt for initial JSON or patch operations.
- **LLM → Extractor**: returns JSON or patch operations.
- **Extractor → Schema Validator**: validates output; on errors, generates targeted JSON patch prompt.
- **Extractor → JSON Patch Engine**: applies patches to build up or update the object.
- **Extractor → LLM (retry)**: repeats until schema validation passes or retry limit reached.
- **Tests/CI → Repo**: run unit tests, evals, and ensure code & LLM workflows behave as expected.

##### Architectural Patterns & Principles
- **Iterative Refinement**: generate small patches rather than full replacements.
- **Separation of Concerns**: core extraction logic is separate from LLM integration (`_validation_node.py`).
- **Plugin‐style Tools**: schemas can be pydantic models, Python functions, or JSON Schemas.
- **Resilience**: automatic retries on validation errors; cassette‐based testing for reproducibility.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Runtime Flow
    UserApp["User Application"]:::runtime
    UserApp -->|"invoke extractor"| CoreOrch

    subgraph "Extractor Core"
        direction TB
        CoreOrch["Core Orchestrator"]:::core
        Base["_base.py"]:::core
        ValidationNode["_validation_node.py"]:::core
        Typed["py.typed"]:::core
        CoreOrch --> Base
        CoreOrch --> ValidationNode
        CoreOrch --> Typed
    end

    %% External Dependencies
    CoreOrch -->|"prompt"| LLMProvider["LLM Provider"]:::external
    LLMProvider -->|"response"| CoreOrch
    CoreOrch -->|"validate"| SchemaValidator["Schema Validator"]:::external
    CoreOrch -->|"apply patch"| JSONPatch["JSON Patch Engine"]:::core
    CoreOrch -->|"retry"| LLMProvider

    %% Testing & CI
    subgraph "Tests & CI"
        direction TB
        CI["GitHub Actions"]:::testing
        subgraph "Workflows"
            direction TB
            TestWF["test.yml"]:::testing
            EvalWF["eval.yml"]:::testing
        end
        subgraph "Unit Tests"
            direction TB
            Unit["tests/unit_tests/"]:::testing
        end
        subgraph "Eval Tests"
            direction TB
            EvalTest["tests/evals/test_evals.py"]:::testing
            EvalCassettes["tests/evals/cassettes/"]:::testing
        end
        subgraph "VCR Cassettes"
            direction TB
            VCR1["tests/cassettes/"]:::testing
        end
        CI --> TestWF
        CI --> EvalWF
        CI --> Unit
        CI --> EvalTest
        CI --> VCR1
    end

    %% Project Configuration
    subgraph "Project Config"
        direction TB
        TOML["pyproject.toml"]:::testing
        Make["Makefile"]:::testing
    end

    %% Click Events
    click UserApp "https://github.com/hinthornw/trustcall/blob/main/README.md"
    click Base "https://github.com/hinthornw/trustcall/blob/main/trustcall/_base.py"
    click ValidationNode "https://github.com/hinthornw/trustcall/blob/main/trustcall/_validation_node.py"
    click Typed "https://github.com/hinthornw/trustcall/blob/main/trustcall/py.typed"
    click TestWF "https://github.com/hinthornw/trustcall/blob/main/.github/workflows/test.yml"
    click EvalWF "https://github.com/hinthornw/trustcall/blob/main/.github/workflows/eval.yml"
    click Unit "https://github.com/hinthornw/trustcall/tree/main/tests/unit_tests/"
    click EvalTest "https://github.com/hinthornw/trustcall/blob/main/tests/evals/test_evals.py"
    click EvalCassettes "https://github.com/hinthornw/trustcall/tree/main/tests/evals/cassettes/"
    click VCR1 "https://github.com/hinthornw/trustcall/tree/main/tests/cassettes/"
    click TOML "https://github.com/hinthornw/trustcall/blob/main/pyproject.toml"
    click Make "https://github.com/hinthornw/trustcall/tree/main/Makefile"

    %% Styles
    classDef runtime fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef core fill:#d4edda,stroke:#333,stroke-width:1px
    classDef external fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef testing fill:#e2e3e5,stroke:#333,stroke-width:1px
```
