# swax10/pytorch-ddp architecture

> Architecture diagram of the GitHub repository swax10/pytorch-ddp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/swax10/pytorch-ddp
- Repository: https://github.com/swax10/pytorch-ddp

## Overview

##### Project type and purpose
- This is a Python-based machine-learning training project.
- It provides two entry points: single-node training (Single_node_training.py) and multi-node (distributed) training (Multi_node_traning.py).
- Core logic lives in model.py (model definition) and dataset.py (data ingestion/processing).

##### File structure
- Top-level scripts:
- Single_node_training.py – launch a local training run
- Multi_node_traning.py – launch a distributed training run across multiple workers/nodes
- Core modules:
- dataset.py – loads and preprocesses data, guided by dataset.json
- model.py – defines the neural net architecture and forward/backward logic
- Configuration/deployment:
- dataset.json – dataset metadata (paths, labels, preprocessing settings)
- pyproject.toml & setup.py – project metadata, dependencies, packaging

##### (empty) README
- No extra architectural description. Rely on code organization to infer components.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Orchestration Layer
    subgraph "Orchestration Layer"
        direction TB
        Single["Single-node Runner<br/>Single_node_training.py"]:::orchestration
        Multi["Distributed Runner<br/>Multi_node_traning.py"]:::orchestration
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Config["Dataset Config<br/>(dataset.json)"]:::data_storage
        DataModule["DataLoader & Preprocessor<br/>(dataset.py)"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        direction TB
        Model["Model Definition & Training<br/>(model.py)"]:::compute
    end

    %% Compute Resources
    subgraph "Compute Resources"
        direction TB
        SingleNode["CPU/GPU (Local)"]:::compute
        Cluster["Cluster Nodes"]:::compute
    end

    %% Storage Layer
    subgraph "Storage"
        direction TB
        Checkpoint["Checkpoints & Logs"]:::storage
    end

    %% Packaging & Dependencies
    subgraph "Packaging & Dependencies"
        direction TB
        Pyproject["pyproject.toml"]:::packaging
        Setup["setup.py"]:::packaging
    end

    %% Control & Data Flows
    Single -->|"reads"| Config
    Single -->|"load data"| DataModule
    Single -->|"init model"| Model
    Single -->|"runs on"| SingleNode

    Multi -->|"reads"| Config
    Multi -->|"load data"| DataModule
    Multi -->|"init model"| Model
    Multi -->|"runs on"| Cluster

    Config -->|"provides metadata"| DataModule
    DataModule -->|"provides batches"| Model
    SingleNode -->|"trains"| Model
    Cluster -->|"trains & sync"| Model
    Cluster -->|"all-reduce"| Cluster

    Model -->|"writes"| Checkpoint

    Pyproject -->|"defines env"| Single
    Pyproject -->|"defines env"| Multi
    Setup -->|"install deps"| Single
    Setup -->|"install deps"| Multi

    %% Click Events
    click Single "https://github.com/swax10/pytorch-ddp/blob/master/Single_node_training.py"
    click Multi "https://github.com/swax10/pytorch-ddp/blob/master/Multi_node_traning.py"
    click Config "https://github.com/swax10/pytorch-ddp/blob/master/dataset.json"
    click DataModule "https://github.com/swax10/pytorch-ddp/blob/master/dataset.py"
    click Model "https://github.com/swax10/pytorch-ddp/blob/master/model.py"
    click Pyproject "https://github.com/swax10/pytorch-ddp/blob/master/pyproject.toml"
    click Setup "https://github.com/swax10/pytorch-ddp/blob/master/setup.py"

    %% Styles
    classDef orchestration fill:#FFA500,stroke:#333,stroke-width:1px;
    classDef data fill:#7CFC00,stroke:#333,stroke-width:1px;
    classDef data_storage fill:#ADFF2F,stroke:#333,stroke-width:1px;
    classDef compute fill:#1E90FF,stroke:#333,stroke-width:1px;
    classDef storage fill:#9370DB,stroke:#333,stroke-width:1px;
    classDef packaging fill:#A9A9A9,stroke:#333,stroke-width:1px;
```
