# pytorch/torchft architecture

> Architecture diagram of the GitHub repository pytorch/torchft, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pytorch/torchft
- Repository: https://github.com/pytorch/torchft

## Overview

##### Main Components
- Lighthouse Server – This is the central coordination entity used to manage replica groups and coordinate fault tolerance actions. It is implemented in Rust as seen from the “src/lighthouse.rs” file and is invoked externally (e.g., via the CLI “torchft_lighthouse”).
- Fault Tolerance Library (Manager and associated modules) – This library implements key routines (in torchft directory) for handling checkpointing, retries, and error recovery on a per-step basis. It includes modules such as:
- Coordination (torchft/coordination.py)
- Manager (torchft/manager.py)
- Process Group (torchft/process_group.py)
- Checkpointing (torchft/checkpointing/*)
- Data handling, futures, and optim (torchft/data.py, torchft/futures.py, torchft/optim.py)
- Training Worker and PyTorch Training Loop – The integration into the user’s training loop is showcased in the “train_ddp.py” example. Workers are enhanced for fault tolerance by wrapping model and optimizer updates with torchft’s manager and fault tolerance logic.
- Parameter Server (optional component) – There is also a fault tolerant parameter server implementation (torchft/parameter_server.py) that follows a similar reconfigurable ProcessGroup model.
- Network Communication & Protobuf Interfaces – The “proto/torchft.proto” indicates use of Protocol Buffers for defining network messages to facilitate communication between the lighthouse server, managers, and the training workers. There are specific modules in torchft for HTTP transport and PG (process group) transport.
- Rust-Python Integration – With build scripts (Cargo.toml, build.rs) and the usage of pyo3/maturin, there is a strong interoperation element between Rust backend components and Python’s torchft library.

##### Relationships and Interactions
- When using the Parameter Server mode, a separate flow is seen where the workers interact with the “Parameter Server” instead of coordinating directly with a lighthouse.

##### Architectural Patterns and Technical Stacks
- That this is a hybrid system design with a clear separation between:
- The coordination layer implemented in Rust (for performance and reliability in network communication)
- The fault tolerance logic integrated within the PyTorch training loops (Python modules)
- The service-oriented and loosely coupled structure supports pluggable modules (e.g., different transport protocols: HTTP vs. PG).
- The use of Protocol Buffers to define communication interfaces reinforces clear contracts between components.
- Fault tolerance is achieved on a per-step granularity, meaning that the system is architected to enable minimal downtime – represented as fast communication loops between Manager and Lighthouse Server.

##### Final Checklist for the Principal Software Engineer
▪ Rust-based components (e.g., Lighthouse Server, underlying network transport)
▪ Python-based components (e.g., Manager, training loop integration)
▪ External services (e.g., External client calls or parameter server)
- Interactions (such as “sends heartbeat”, “requests recovery”, “shares checkpoint data”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Rust Components
    subgraph "Rust Components"
        LS["Lighthouse Server (Rust)"]:::rust
        Integration["Rust-Python Integration"]:::rust
    end

    %% Python Components
    subgraph "Python Components"
        subgraph "Fault Tolerance Library"
            Coord["Coordination Module"]:::python
            Manager["Manager Module"]:::python
            PG["Process Group Module"]:::python
            CP["Checkpointing Modules"]:::python
            Data["Data Handling Module"]:::python
            Futures["Futures Module"]:::python
            Optim["Optim Module"]:::python
        end
        Train["Training Worker (PyTorch Training Loop)"]:::python
        Param["Parameter Server (Optional)"]:::python
        Proto["Network Communication & Protobuf Interfaces"]:::python
    end

    %% Connections between components
    LS -->|"coordination_signals"| Manager
    Manager -->|"sends_recovery"| LS
    Train -->|"invokes_faulttolib"| Manager
    Manager -->|"communicates_with"| PG
    Manager -->|"handles_checkpoint"| CP
    CP -->|"transport_comm"| Proto
    Manager -->|"process_data"| Data
    Manager -->|"async_calls"| Futures
    Manager -->|"performs_optim"| Optim
    Manager -->|"network_msgs"| Proto
    LS -->|"network_msgs"| Proto
    Train -->|"optionally_interacts"| Param
    Manager -->|"coordinates"| Param
    Integration -->|"bridges_component"| LS
    Integration -->|"bridges_component"| Manager

    %% Click Events
    click LS "https://github.com/pytorch/torchft/blob/main/src/lighthouse.rs"
    click LS "https://github.com/pytorch/torchft/blob/main/src/bin/lighthouse.rs"
    click Coord "https://github.com/pytorch/torchft/blob/main/torchft/coordination.py"
    click Manager "https://github.com/pytorch/torchft/blob/main/torchft/manager.py"
    click PG "https://github.com/pytorch/torchft/blob/main/torchft/process_group.py"
    click CP "https://github.com/pytorch/torchft/tree/main/torchft/checkpointing/"
    click Data "https://github.com/pytorch/torchft/blob/main/torchft/data.py"
    click Futures "https://github.com/pytorch/torchft/blob/main/torchft/futures.py"
    click Optim "https://github.com/pytorch/torchft/blob/main/torchft/optim.py"
    click Train "https://github.com/pytorch/torchft/blob/main/train_ddp.py"
    click Param "https://github.com/pytorch/torchft/blob/main/torchft/parameter_server.py"
    click Proto "https://github.com/pytorch/torchft/blob/main/proto/torchft.proto"
    click Integration "https://github.com/pytorch/torchft/blob/main/build.rs"
    click Integration "https://github.com/pytorch/torchft/blob/main/Cargo.toml"
    click Integration "https://github.com/pytorch/torchft/blob/main/pyproject.toml"

    %% Styles
    classDef rust fill:#ffadad,stroke:#333,stroke-width:2px;
    classDef python fill:#caffbf,stroke:#333,stroke-width:2px;
```
