# guidance-ai/guidance architecture

> Architecture diagram of the GitHub repository guidance-ai/guidance, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/guidance-ai/guidance
- Repository: https://github.com/guidance-ai/guidance

## Overview

Based on our analysis of both the file tree and the README, Guidance is a Python library that provides a programming paradigm to “steer” language models. It does so by letting developers write “guidance functions” (decorated Python functions) that interleave static text with controlled generation calls. The project supports multiple backends (Transformers, llama.cpp, OpenAI, Vertex AI, etc.), constrained generation (through regex, CFGs, selects), and even tool integration (for automatic calls like a calculator). There is also a client-side component (implemented in Svelte) and a comprehensive set of notebooks, tests, and even packaging for web widgets (in the “stitch” package).

##### Major Components
- Core Guidance Library: This is the heart of the project and resides in the “guidance” directory. Its submodules include components for parsing and executing guidance programs (_ast, _grammar, _parser, _schema, _guidance, _utils), metrics gathering, visualizations (trace, visual), and integration with various models (e.g. models, registry, library).
- Tooling / Domain-specific Grammars & Extensions: Submodules in “guidance/library” (audio, image, json, etc.), along with the “stitch” package under “packages/python/stitch,” provide encapsulated reusable functionalities and possibly UI widgets.
- Client Interface: Located in “client/graphpaper-inline”, which appears to be a Svelte-based frontend for interactive use and demonstration of guidance functionality.
- Documentation and Notebooks: The “docs” and “notebooks” directories provide examples, tutorials, and API documentation that illustrate how the library is used for constrained generation, stateful control, and tool interleaving.
- Testing & CI: There is a comprehensive testing suite (in “tests”) and several CI configurations (in “.github/workflows”) that ensure proper functioning across various components.

##### Relationships and Interactions
- The Core Guidance Library sits at the center and orchestrates generation logic. It uses control structures (e.g., the @guidance decorator) to build “lm” objects that capture prompt state.
- Tools and Callables: Mark a subsystem (or set of components) within the library where tools (like the calculator tool) are defined and integrated seamlessly via automatic interruption and resumption of generation.
- Documentation, Notebooks, and Tests should be represented as supporting artifacts that either feed into education or validate the product, without being “runtime” components.

##### Architectural Patterns and Design Principles
- Modular and Pluggable Architecture: Emphasize that the project is broken down into independent modules (guidance/_* modules, models, library) that are loosely coupled. This supports multiple backends and custom tooling.
- Separation of Concerns: The frontend client, core generation logic, model integrations, and extension libraries are implemented in separate directories, each with well-defined roles.
- Stateful Control: Highlight the state management approach (immutable “lm” objects) that allows fine-grained control of text generation and interleaving of generated text with static text.
- Tool Invocation Mechanism: Illustrate the design pattern where generation is interrupted for tool calls (using specific grammars) and then resumed, ensuring that control and generation are interleaved seamlessly.
– Python for the core library and scripting
– JavaScript/Svelte (in client/graphpaper-inline) for the user interface
– CI/CD scripts (GitHub Actions workflows)
– Documentation using reStructuredText (with ReadTheDocs)

##### Specific Elements
– Parsing & Grammar ([_ast, _grammar, _parser, _schema])
– Control & Generation (_guidance, gen, select, capture)
– Tracing & Visualizations (trace, visual)
– Different adapters (OpenAI, LlamaCpp, Transformers, Vertex AI) coming off the main guidance module.
– Showcase the frontend “graphpaper-inline” client.
– Highlight modules like the calculator tool, and grammars for constrained generation (select, regex, CFGs).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Guidance Core Library Subgraph
    subgraph "Guidance Core Library"
        direction TB
        CORE1["Parsing & Grammar\n(_ast.py, _grammar.py, _parser.py, _schema.py)"]:::core
        CORE2["Control & Generation\n(_guidance.py, gen, select, capture)"]:::core
        CORE3["Tracing & Visualizations\n(trace, visual)"]:::core
        CORE1 --> CORE2 --> CORE3
    end

    %% Model Integration & Backend Abstraction
    MI["Model Integration & Backend Abstraction\n(OpenAI, LlamaCpp, Transformers, Vertex AI, broken_models)"]:::integration

    %% Tooling & Extensions Subgraph
    subgraph "Tooling & Extensions"
        direction TB
        TL1["Tools & Grammars\n(guidance/library)"]:::tooling
        TL2["Web Widgets Packaging\n(packages/python/stitch)"]:::tooling
        TL1 --> TL2
    end

    %% Client Interface
    CI["Client Interface\n(Graphpaper-Inline Svelte)"]:::client

    %% Documentation & Samples Subgraph
    subgraph "Documentation & Samples"
        direction TB
        DOC1["Documentation\n(docs)"]:::support
        DOC2["Notebooks\n(notebooks)"]:::support
        DOC1 --> DOC2
    end

    %% Testing & CI Subgraph
    subgraph "Testing & CI"
        direction TB
        TST["Tests\n(tests)"]:::support
        CI_CD["CI Workflows\n(.github/workflows)"]:::support
        TST --> CI_CD
    end

    %% Connections
    CORE2 -->|"LM_APIcalls"| MI
    CORE2 -->|"ToolInvocation"| TL1
    CI -->|"UserInput"| CORE2
    CORE2 -->|"Results"| CI
    DOC1 -->|"Examples"| CORE2
    TST -->|"Validation"| CORE2
    CI_CD -->|"Validation"| CORE2

    %% Click Events
    click CORE1 "https://github.com/guidance-ai/guidance/tree/main/guidance"
    click MI "https://github.com/guidance-ai/guidance/tree/main/guidance/models"
    click TL1 "https://github.com/guidance-ai/guidance/tree/main/guidance/library"
    click TL2 "https://github.com/guidance-ai/guidance/tree/main/packages/python/stitch"
    click CI "https://github.com/guidance-ai/guidance/tree/main/client/graphpaper-inline"
    click DOC1 "https://github.com/guidance-ai/guidance/tree/main/docs"
    click DOC2 "https://github.com/guidance-ai/guidance/tree/main/notebooks"
    click TST "https://github.com/guidance-ai/guidance/tree/main/tests"
    click CI_CD "https://github.com/guidance-ai/guidance/tree/main/.github/workflows"

    %% Styles
    classDef core fill:#ccf,stroke:#333,stroke-width:2px;
    classDef integration fill:#cfc,stroke:#333,stroke-width:2px;
    classDef tooling fill:#fcf,stroke:#333,stroke-width:2px;
    classDef client fill:#cff,stroke:#333,stroke-width:2px;
    classDef support fill:#ffc,stroke:#333,stroke-width:2px;
```
