# pytorch/torchtune architecture

> Architecture diagram of the GitHub repository pytorch/torchtune, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pytorch/torchtune
- Repository: https://github.com/pytorch/torchtune

## Overview

##### Main Components

a) Core Library Components:
- Model Management System
- Model implementations (Llama, Gemma, Mistral, etc.)
- Model builders and component builders
- Tokenizers
- Training System
- Finetuning recipes (Full, LoRA, QLoRA, DPO, PPO)
- Distributed training support
- Memory optimization features
- Dataset Management
- Dataset implementations
- Data collation and processing
- Message transforms
- Configuration System
- YAML config parsing
- Config validation
- Instantiation logic
- CLI Interface
- Command processing
- Recipe running
- Model downloading

b) External Integrations:
- PyTorch Core (FSDP2, compile)
- Hugging Face (Hub, Datasets)
- Weights & Biases / Comet (Logging)
- EleutherAI (Evaluation)
- torchao (Quantization)
- bitsandbytes (Optimizers)

##### Relationship Flow

- CLI receiving user commands and interfacing with other components
- Config system feeding into model, training, and dataset components
- Training system orchestrating model, dataset, and optimization components
- External integrations connecting at appropriate points

- [External Services]
- ↑

**[CLI Interface] → [Config System] → [Training System] ← [Memory Optimizations]**
- ↓               ↓                ↑    ↑

- ↓               ↓                |    |
**[Model Downloads] [Config Validation]   |    |**

**[Model Management] ----------------→ [Models]**
- ↑                                ↑

**[Model Components] ---------------→ [Datasets]**

##### Implementation Details to Highlight

- Memory optimization techniques (activation checkpointing, offloading)
- Multiple training paradigms (distributed, single-device)
- Model family support structure
- Dataset format flexibility
- Configuration system's extensibility

##### Key Architectural Patterns

- Factory pattern for model creation
- Strategy pattern for different training approaches
- Observer pattern for logging and metrics
- Builder pattern for configuration
- Repository pattern for model management

This architecture reflects torchtune's focus on providing a flexible, efficient framework for LLM finetuning while maintaining integration with the broader PyTorch ecosystem.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph External["External Services"]
        PyTorch["PyTorch Core"]:::external
        HF["HuggingFace Hub"]:::external
        WandB["Weights & Biases"]:::external
        Eleuther["EleutherAI"]:::external
        TorchAO["torchao"]:::external
        BNB["bitsandbytes"]:::external
    end

    subgraph CLI["CLI Interface"]
        Commands["Command Processing"]:::cli
        Recipes["Recipe Running"]:::cli
        Downloads["Model Downloads"]:::cli
    end

    subgraph Config["Configuration System"]
        Parser["Config Parsing"]:::config
        Validator["Config Validation"]:::config
        Instantiator["Instantiation Logic"]:::config
    end

    subgraph Training["Training System"]
        Finetuning["Finetuning Recipes"]:::training
        DistTraining["Distributed Training"]:::training
        MemOpt["Memory Optimization"]:::training
        Checkpoint["Checkpointing"]:::training
    end

    subgraph Models["Model Management"]
        Llama["Llama Implementation"]:::model
        Gemma["Gemma Implementation"]:::model
        Mistral["Mistral Implementation"]:::model
        PEFT["PEFT Components"]:::model
    end

    subgraph RLHF["RLHF Components"]
        DPO["DPO Implementation"]:::rlhf
        PPO["PPO Implementation"]:::rlhf
        Rewards["Reward Modeling"]:::rlhf
    end

    subgraph Data["Dataset Management"]
        Datasets["Core Dataset Components"]:::data
        Collation["Data Collation"]:::data
        Messages["Message Transforms"]:::data
        Tokenizers["Tokenizer Components"]:::data
    end

    Commands --> Parser
    Parser --> Validator
    Validator --> Instantiator
    Instantiator --> Training
    Downloads --> Models
    Recipes --> Training

    Training --> Models
    Training --> Data
    Training --> RLHF
    
    Models --> Data
    RLHF --> Models

    %% External connections
    Training --> PyTorch
    Models --> HF
    Training --> WandB
    Training --> Eleuther
    Models --> TorchAO
    Training --> BNB

    %% Click events
    click Commands "https://github.com/pytorch/torchtune/blob/main//torchtune/_cli/tune.py"
    click Recipes "https://github.com/pytorch/torchtune/blob/main//torchtune/_cli/run.py"
    click Downloads "https://github.com/pytorch/torchtune/blob/main//torchtune/_cli/download.py"
    click Parser "https://github.com/pytorch/torchtune/blob/main//torchtune/config/_parse.py"
    click Validator "https://github.com/pytorch/torchtune/blob/main//torchtune/config/_validate.py"
    click Instantiator "https://github.com/pytorch/torchtune/blob/main//torchtune/config/_instantiate.py"
    click MemOpt "https://github.com/pytorch/torchtune/blob/main//torchtune/training/memory.py"
    click DistTraining "https://github.com/pytorch/torchtune/blob/main//torchtune/training/_distributed.py"
    click Checkpoint "https://github.com/pytorch/torchtune/tree/main//torchtune/training/checkpointing/"
    click Llama "https://github.com/pytorch/torchtune/tree/main//torchtune/models/llama3/"
    click Gemma "https://github.com/pytorch/torchtune/tree/main//torchtune/models/gemma/"
    click Mistral "https://github.com/pytorch/torchtune/tree/main//torchtune/models/mistral/"
    click DPO "https://github.com/pytorch/torchtune/blob/main//torchtune/rlhf/loss/dpo.py"
    click PPO "https://github.com/pytorch/torchtune/blob/main//torchtune/rlhf/loss/ppo.py"
    click Rewards "https://github.com/pytorch/torchtune/blob/main//torchtune/rlhf/rewards.py"
    click Datasets "https://github.com/pytorch/torchtune/tree/main//torchtune/datasets/"
    click Collation "https://github.com/pytorch/torchtune/blob/main//torchtune/data/_collate.py"
    click Messages "https://github.com/pytorch/torchtune/blob/main//torchtune/data/_messages.py"
    click Tokenizers "https://github.com/pytorch/torchtune/tree/main//torchtune/modules/tokenizers/"

    %% Styling
    classDef external fill:#90EE90
    classDef cli fill:#ADD8E6
    classDef config fill:#DDA0DD
    classDef training fill:#FFB6C1
    classDef model fill:#F0E68C
    classDef rlhf fill:#FFA07A
    classDef data fill:#87CEEB

    %% Legend
    subgraph Legend
        L1["External Services"]:::external
        L2["CLI Components"]:::cli
        L3["Config Components"]:::config
        L4["Training Components"]:::training
        L5["Model Components"]:::model
        L6["RLHF Components"]:::rlhf
        L7["Data Components"]:::data
    end
```
