# lesc-ufv/ddp-gradient-compression architecture

> Architecture diagram of the GitHub repository lesc-ufv/ddp-gradient-compression, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lesc-ufv/ddp-gradient-compression
- Repository: https://github.com/lesc-ufv/ddp-gradient-compression

## Overview

This is an experimental PyTorch DDP gradient-compression lab. A user launches the training runner, which configures distributed execution, prepares data and a model, then trains with a selected communication hook; rank 0 evaluates and writes epoch metrics. The main capabilities are dense and FP16 baselines, Top-K with error feedback, PowerSGD, and optional ACCL collectives. The graph reflects README-described DDP behavior and shown source calls; unsampled evaluation and ACCL internals are represented only at the supported subsystem level.

Main entry points: [`src/ddp_gradient_compression/train.py`](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/train.py) (Training runner), [`src/ddp_gradient_compression/hooks/factory.py`](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/factory.py) (Hook selection), [`src/ddp_gradient_compression/backends/accl.py`](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl.py) (ACCL process group), [`src/ddp_gradient_compression/backends/accl_runtime.py`](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl_runtime.py) (PyACCL adapter)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Training"]
  node_runner["Training runner<br/>[train.py]"]
  node_config["Train configuration<br/>[config.py]"]
  node_data["Dataset and loaders<br/>[data.py]"]
  node_model["Model builder<br/>[models.py]"]
  node_evaluation["Epoch evaluation<br/>[train.py]"]
end

subgraph group_compression["Compression hooks"]
  node_hook_factory["Hook selection<br/>[factory.py]"]
  node_topk["Top-K hook<br/>[topk.py]"]
  node_powersgd["PowerSGD hook<br/>[powersgd.py]"]
  node_baseline_hooks["Dense and FP16<br/>[factory.py]"]
end

subgraph group_distributed["Distributed backends"]
  node_ddp["PyTorch DDP"]
  node_distributed_context["Distributed setup<br/>[distributed.py]"]
  node_gloo["Gloo collectives"]
  node_accl_group["ACCL process group<br/>[accl.py]"]
  node_accl_config["Rank-device config<br/>[accl_config.py]"]
  node_accl_runtime["PyACCL adapter<br/>[accl_runtime.py]"]
end

subgraph group_results["Results"]
  node_metrics["CSV metrics<br/>[metrics.py]"]
end

node_user(("Experimenter"))
node_fpga["ACCL device or emulator"]

node_user -->|"launches"| node_runner
node_runner -->|"parses options"| node_config
node_runner -->|"initializes ranks"| node_distributed_context
node_runner -->|"prepares loaders"| node_data
node_runner -->|"builds model"| node_model
node_runner -->|"wraps model"| node_ddp
node_runner -->|"builds hook"| node_hook_factory
node_ddp -->|"invokes registered hook"| node_hook_factory
node_hook_factory -->|"selects"| node_topk
node_hook_factory -->|"selects"| node_powersgd
node_hook_factory -->|"selects"| node_baseline_hooks
node_topk -->|"gathers sparse payload"| node_gloo
node_powersgd -->|"reduces factors"| node_gloo
node_distributed_context -->|"registers optional backend"| node_accl_group
node_accl_group -->|"loads rank mapping"| node_accl_config
node_accl_group -->|"creates runtime"| node_accl_runtime
node_accl_runtime -->|"uses configuration"| node_accl_config
node_accl_runtime -->|"runs collectives"| node_fpga
node_runner -->|"evaluates epochs"| node_evaluation
node_runner -->|"writes epoch metrics"| node_metrics

click node_runner "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/train.py"
click node_config "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/config.py"
click node_data "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/data.py"
click node_model "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/models.py"
click node_hook_factory "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/factory.py"
click node_topk "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/topk.py"
click node_powersgd "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/powersgd.py"
click node_baseline_hooks "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/factory.py"
click node_distributed_context "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/distributed.py"
click node_accl_group "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl.py"
click node_accl_config "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl_config.py"
click node_accl_runtime "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl_runtime.py"
click node_evaluation "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/train.py"
click node_metrics "https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/metrics.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_config,node_data,node_model,node_evaluation toneBlue
class node_hook_factory,node_topk,node_powersgd,node_baseline_hooks toneAmber
class node_ddp,node_distributed_context,node_gloo,node_accl_group,node_accl_config,node_accl_runtime toneMint
class node_metrics toneRose
class node_user,node_fpga toneIndigo
```

## Components

### Training

- [Training runner](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/train.py): `src/ddp_gradient_compression/train.py`
- [Train configuration](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/config.py): `src/ddp_gradient_compression/config.py`
- [Dataset and loaders](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/data.py): `src/ddp_gradient_compression/data.py`
- [Model builder](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/models.py): `src/ddp_gradient_compression/models.py`
- [Epoch evaluation](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/train.py): `src/ddp_gradient_compression/train.py`

### Compression hooks

- [Hook selection](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/factory.py): `src/ddp_gradient_compression/hooks/factory.py`
- [Top-K hook](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/topk.py): `src/ddp_gradient_compression/hooks/topk.py`
- [PowerSGD hook](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/powersgd.py): `src/ddp_gradient_compression/hooks/powersgd.py`
- [Dense and FP16](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/hooks/factory.py): `src/ddp_gradient_compression/hooks/factory.py`

### Distributed backends

- **PyTorch DDP**
- [Distributed setup](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/distributed.py): `src/ddp_gradient_compression/distributed.py`
- **Gloo collectives**
- [ACCL process group](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl.py): `src/ddp_gradient_compression/backends/accl.py`
- [Rank-device config](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl_config.py): `src/ddp_gradient_compression/backends/accl_config.py`
- [PyACCL adapter](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/backends/accl_runtime.py): `src/ddp_gradient_compression/backends/accl_runtime.py`

### Results

- [CSV metrics](https://github.com/lesc-ufv/ddp-gradient-compression/blob/main/src/ddp_gradient_compression/metrics.py): `src/ddp_gradient_compression/metrics.py`

### Other components

- **Experimenter**
- **ACCL device or emulator**

## Connections

- Experimenter → Training runner: launches (evidence: `README.md`)
- Training runner → Train configuration: parses options (evidence: `src/ddp_gradient_compression/train.py`)
- Training runner → Distributed setup: initializes ranks (evidence: `src/ddp_gradient_compression/train.py`)
- Training runner → Dataset and loaders: prepares loaders (evidence: `src/ddp_gradient_compression/train.py`)
- Training runner → Model builder: builds model (evidence: `src/ddp_gradient_compression/train.py`)
- Training runner → PyTorch DDP: wraps model (evidence: `src/ddp_gradient_compression/train.py`)
- Training runner → Hook selection: builds hook (evidence: `src/ddp_gradient_compression/train.py`)
- PyTorch DDP → Hook selection: invokes registered hook (evidence: `README.md`)
- Hook selection → Top-K hook: selects (evidence: `src/ddp_gradient_compression/hooks/factory.py`)
- Hook selection → PowerSGD hook: selects (evidence: `src/ddp_gradient_compression/hooks/factory.py`)
- Hook selection → Dense and FP16: selects (evidence: `src/ddp_gradient_compression/hooks/factory.py`)
- Top-K hook → Gloo collectives: gathers sparse payload (evidence: `src/ddp_gradient_compression/hooks/topk.py`)
- PowerSGD hook → Gloo collectives: reduces factors (evidence: `src/ddp_gradient_compression/hooks/powersgd.py`)
- Distributed setup → ACCL process group: registers optional backend (evidence: `src/ddp_gradient_compression/distributed.py`)
- ACCL process group → Rank-device config: loads rank mapping (evidence: `src/ddp_gradient_compression/backends/accl.py`)
- ACCL process group → PyACCL adapter: creates runtime (evidence: `src/ddp_gradient_compression/backends/accl.py`)
- PyACCL adapter → Rank-device config: uses configuration (evidence: `src/ddp_gradient_compression/backends/accl_runtime.py`)
- PyACCL adapter → ACCL device or emulator: runs collectives (evidence: `src/ddp_gradient_compression/backends/accl_runtime.py`)
- Training runner → Epoch evaluation: evaluates epochs (evidence: `README.md`)
- Training runner → CSV metrics: writes epoch metrics (evidence: `src/ddp_gradient_compression/train.py`)
