# 567-labs/instructor architecture

> Architecture diagram of the GitHub repository 567-labs/instructor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/567-labs/instructor
- Repository: https://github.com/567-labs/instructor

## Overview

##### Project Type and Purpose
- Instructor is an open-source Python library (not a deployed application) that provides a unified, Pydantic-based API for generating and validating structured outputs from LLMs.
- It wraps many provider SDKs (OpenAI, Anthropic, Google Gemini, Mistral, Cohere, etc.), offering features like retries, hooks, streaming, and type inference.

##### High-Level Architectural Components
A. Core Library (instructor/)
- Client factory (from_provider, from_openai, etc.)
- Provider adapters (client_openai.py, client_anthropic.py, etc.)
- DSL & templating modules (instructor/dsl, templating.py)
- Response processing (validation via Pydantic, retry logic, streaming partials/iterables)
- Hooks subsystem (hook registration and event dispatch)
- Utility modules (utils.py, process_response.py, retry.py, patch.py)

B. Command-Line Interface (instructor/cli/)
- CLI entrypoints (“instructor jobs”, “instructor files”, “instructor usage”)
- Job management, file uploads, usage metrics wrappers around core client

C. Examples & Documentation
- examples/ – Cookbook‐style scripts demonstrating various use cases
- docs/ + mkdocs.yml – MkDocs static site for API reference, tutorials, concept guides
- Build scripts (build_mkdocs.sh, make_sitemap.py)

D. Testing & CI
- tests/ – Unit tests and eval suites across providers
- .github/workflows – CI pipelines for linting (ruff), type checking (pyright), tests, docs validation

##### Relationships and Data Flows
- User code (or CLI) invokes Instructor client →
- Client factory selects appropriate provider adapter →
- Adapter calls underlying LLM SDK (e.g., OpenAI.ChatCompletion) →
- Responses streamed or batched back →
- Instructor applies retry policies, Pydantic validation, hooks →
- Final structured models returned to user

##### Key Architectural Patterns
- Adapter pattern for provider integrations
- Decorator/Monkey-patching for injecting type inference into underlying SDKs
- Strategy pattern for retries and validation strategies
- Event dispatcher for hooks
- Modular layering: Core → Adapters → External services

##### Technologies and Frameworks
- Python 3, Pydantic for models/validation
- Underlying SDKs: openai, anthropic, google-generativeai, cohere, etc.
- CLI: click or argparse (instructor/cli)
- Documentation: MkDocs with YAML config, pre-commit hooks for docs checks
- Testing: pytest, uv for dependency management, ruff for linting, pyright for type checking

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    User["User Code / CLI"]:::user

    %% Core Library Subgraph
    subgraph "Instructor Core" 
        direction TB
        CF["Client Factory"]:::core
        subgraph "Provider Adapters" 
            direction TB
            OA["OpenAI Adapter"]:::adapter
            AA["Anthropic Adapter"]:::adapter
            GA["Google Gemini Adapter"]:::adapter
            MA["Mistral Adapter"]:::adapter
            CA["Cohere Adapter"]:::adapter
            BA["Bedrock Adapter"]:::adapter
            CeA["Cerebras Adapter"]:::adapter
            FA["Fireworks Adapter"]:::adapter
            GnA["GenAI Adapter"]:::adapter
            GrA["GROQ Adapter"]:::adapter
            PA["Perplexity Adapter"]:::adapter
            VA["Vertex AI Adapter"]:::adapter
            WA["Writer Adapter"]:::adapter
        end
        subgraph "DSL & Templating"
            direction TB
            DSL["DSL Modules"]:::dsl
            TE["Templating Engine"]:::dsl
        end
        subgraph "Response Processing"
            direction TB
            RH["Response Handler"]:::processing
            RL["Retry Logic"]:::processing
            PU["Patch Utilities"]:::processing
        end
        HS["Hooks Subsystem"]:::hooks
        subgraph "Utilities"
            direction TB
            GU["General Utilities"]:::utils
            VM["Validation & Models"]:::utils
            FC["Function Call Support"]:::utils
            ReA["Reask Logic"]:::utils
            BS["Batching Support"]:::utils
            MM["Mode Management"]:::utils
            MD["Multimodal Handling"]:::utils
            DS["Distillation Support"]:::utils
        end
    end

    %% CLI Subgraph
    subgraph "Command-Line Interface" 
        direction TB
        CLI["CLI Entry Point"]:::cli
        CJ["Job Management"]:::cli
        CFU["File Uploads"]:::cli
        UM["Usage Metrics"]:::cli
        CB["Batch Commands"]:::cli
        DH["Deprecated Hub Commands"]:::cli
    end

    %% Examples & Documentation
    subgraph "Examples & Documentation"
        direction TB
        EX["Cookbook Scripts"]:::support
        DOC["API Reference & Tutorials"]:::support
        MD["MkDocs Build Scripts"]:::support
    end

    %% Testing & CI
    subgraph "Testing & CI"
        direction TB
        TST["Unit & Integration Tests"]:::support
        CI["CI Workflows"]:::support
    end

    %% External Services
    subgraph "External LLM Services"
        direction TB
        OpenAI_API["OpenAI API"]:::external
        Anthropic_API["Anthropic API"]:::external
        Gemini_API["Google Gemini API"]:::external
        Mistral_API["Mistral API"]:::external
        Cohere_API["Cohere API"]:::external
        Bedrock_API["AWS Bedrock API"]:::external
        Cerebras_API["Cerebras API"]:::external
        Fireworks_API["Fireworks AI API"]:::external
        GenAI_API["Azure GenAI API"]:::external
        GROQ_API["GROQ AI API"]:::external
        Perplexity_API["Perplexity API"]:::external
        VertexAI_API["Vertex AI API"]:::external
        Writer_API["Writer API"]:::external
    end

    %% Runtime Flow
    User -->|"invoke"| CF
    CF -->|"select adapter"| OA & AA & GA & MA & CA & BA & CeA & FA & GnA & GrA & PA & VA & WA
    OA -->|calls| OpenAI_API
    AA -->|calls| Anthropic_API
    GA -->|calls| Gemini_API
    MA -->|calls| Mistral_API
    CA -->|calls| Cohere_API
    BA -->|calls| Bedrock_API
    CeA -->|calls| Cerebras_API
    FA -->|calls| Fireworks_API
    GnA -->|calls| GenAI_API
    GrA -->|calls| GROQ_API
    PA -->|calls| Perplexity_API
    VA -->|calls| VertexAI_API
    WA -->|calls| Writer_API

    OpenAI_API & Anthropic_API & Gemini_API & Mistral_API & Cohere_API & Bedrock_API & Cerebras_API & Fireworks_API & GenAI_API & GROQ_API & Perplexity_API & VertexAI_API & Writer_API --> RH
    RH --> RL
    RL --> HS
    HS --> VM
    HS --> FC
    HS --> GU
    HS --> ReA
    HS --> BS
    HS --> MM
    HS --> MM
    HS --> DS
    RH -->|"return structured model"| User

    %% CLI Flow
    User -->|"CLI command"| CLI
    CLI --> CFU
    CLI --> CJ
    CLI --> UM
    CLI --> CB
    CLI --> DH
    CLI --> CF

    %% Support Tools
    User -->|"run examples"| EX
    EX --> CF
    User -->|"view docs"| DOC
    MD --> DOC

    CI --> TST
    CI --> EX
    CI --> DOC

    %% Click Events for component mapping
    click CF "https://github.com/567-labs/instructor/blob/main/instructor/auto_client.py"
    click CF "https://github.com/567-labs/instructor/blob/main/instructor/client.py"
    click OA "https://github.com/567-labs/instructor/blob/main/instructor/client_openai.py"
    click AA "https://github.com/567-labs/instructor/blob/main/instructor/client_anthropic.py"
    click GA "https://github.com/567-labs/instructor/blob/main/instructor/client_gemini.py"
    click MA "https://github.com/567-labs/instructor/blob/main/instructor/client_mistral.py"
    click CA "https://github.com/567-labs/instructor/blob/main/instructor/client_cohere.py"
    click BA "https://github.com/567-labs/instructor/blob/main/instructor/client_bedrock.py"
    click CeA "https://github.com/567-labs/instructor/blob/main/instructor/client_cerebras.py"
    click FA "https://github.com/567-labs/instructor/blob/main/instructor/client_fireworks.py"
    click GnA "https://github.com/567-labs/instructor/blob/main/instructor/client_genai.py"
    click GrA "https://github.com/567-labs/instructor/blob/main/instructor/client_groq.py"
    click PA "https://github.com/567-labs/instructor/blob/main/instructor/client_perplexity.py"
    click VA "https://github.com/567-labs/instructor/blob/main/instructor/client_vertexai.py"
    click WA "https://github.com/567-labs/instructor/blob/main/instructor/client_writer.py"
    click DSL "https://github.com/567-labs/instructor/tree/main/instructor/dsl/"
    click TE "https://github.com/567-labs/instructor/blob/main/instructor/templating.py"
    click RH "https://github.com/567-labs/instructor/blob/main/instructor/process_response.py"
    click RL "https://github.com/567-labs/instructor/blob/main/instructor/retry.py"
    click PU "https://github.com/567-labs/instructor/blob/main/instructor/patch.py"
    click HS "https://github.com/567-labs/instructor/blob/main/instructor/hooks.py"
    click GU "https://github.com/567-labs/instructor/blob/main/instructor/utils.py"
    click VM "https://github.com/567-labs/instructor/blob/main/instructor/validators.py"
    click VM "https://github.com/567-labs/instructor/blob/main/instructor/models.py"
    click FC "https://github.com/567-labs/instructor/blob/main/instructor/function_calls.py"
    click ReA "https://github.com/567-labs/instructor/blob/main/instructor/reask.py"
    click BS "https://github.com/567-labs/instructor/blob/main/instructor/batch.py"
    click MM "https://github.com/567-labs/instructor/blob/main/instructor/mode.py"
    click MM "https://github.com/567-labs/instructor/blob/main/instructor/multimodal.py"
    click DS "https://github.com/567-labs/instructor/blob/main/instructor/distil.py"
    click CLI "https://github.com/567-labs/instructor/blob/main/instructor/cli/cli.py"
    click CJ "https://github.com/567-labs/instructor/blob/main/instructor/cli/jobs.py"
    click CFU "https://github.com/567-labs/instructor/blob/main/instructor/cli/files.py"
    click UM "https://github.com/567-labs/instructor/blob/main/instructor/cli/usage.py"
    click CB "https://github.com/567-labs/instructor/blob/main/instructor/cli/batch.py"
    click DH "https://github.com/567-labs/instructor/blob/main/instructor/cli/deprecated_hub.py"
    click EX "https://github.com/567-labs/instructor/tree/main/examples/"
    click DOC "https://github.com/567-labs/instructor/tree/main/docs/"
    click MD "https://github.com/567-labs/instructor/blob/main/mkdocs.yml"
    click MD "https://github.com/567-labs/instructor/blob/main/build_mkdocs.sh"
    click MD "https://github.com/567-labs/instructor/blob/main/make_sitemap.py"
    click TST "https://github.com/567-labs/instructor/tree/main/tests/"
    click CI "https://github.com/567-labs/instructor/tree/main/.github/workflows/"

    %% Styles
    classDef core fill:#FFD580,stroke:#CC9A00,color:#000
    classDef adapter fill:#FFB3B3,stroke:#CC0000,color:#000
    classDef dsl fill:#C2E0FF,stroke:#005B9A,color:#000
    classDef processing fill:#E2F0CB,stroke:#7BA23F,color:#000
    classDef hooks fill:#EAD1DC,stroke:#9A497A,color:#000
    classDef utils fill:#D9D2E9,stroke:#5E4B8B,color:#000
    classDef cli fill:#BDE4FF,stroke:#0071BC,color:#000
    classDef support fill:#F4CCCC,stroke:#C00,color:#000
    classDef external fill:#D9EAD3,stroke:#6AA84F,color:#000
    classDef user fill:#FCE5CD,stroke:#DD7E6B,color:#000
```
