How lesc-ufv/ddp-gradient-compression works
This is an experimental PyTorch DDP gradient-compression lab. A user launches the training runner, which configures distributed execution, prepares data and a model, then trains with a selected communication hook; rank 0 evaluates and writes epoch metrics. The main capabilities are dense and FP16 baselines, Top-K with error feedback, PowerSGD, and optional ACCL collectives. The graph reflects README-described DDP behavior and shown source calls; unsampled evaluation and ACCL internals are represented only at the supported subsystem level.
Main entry points: train.py Training runner · factory.py Hook selection · accl.py ACCL process group · accl_runtime.py PyACCL adapter
Components 18
Training
- Training runner
src/ddp_gradient_compression/train.py - Train configuration
src/ddp_gradient_compression/config.py - Dataset and loaders
src/ddp_gradient_compression/data.py - Model builder
src/ddp_gradient_compression/models.py - Epoch evaluation
src/ddp_gradient_compression/train.py
Compression hooks
- Hook selection
src/ddp_gradient_compression/hooks/factory.py - Top-K hook
src/ddp_gradient_compression/hooks/topk.py - PowerSGD hook
src/ddp_gradient_compression/hooks/powersgd.py - Dense and FP16
src/ddp_gradient_compression/hooks/factory.py
Distributed backends
- PyTorch DDP
- Distributed setup
src/ddp_gradient_compression/distributed.py - Gloo collectives
- ACCL process group
src/ddp_gradient_compression/backends/accl.py - Rank-device config
src/ddp_gradient_compression/backends/accl_config.py - PyACCL adapter
src/ddp_gradient_compression/backends/accl_runtime.py
Results
- CSV metrics
src/ddp_gradient_compression/metrics.py
Other components
- Experimenter
- ACCL device or emulator
Connections 20
- Experimenter → Training runner: launches
README.md - Training runner → Train configuration: parses options
src/ddp_gradient_compression/train.py - Training runner → Distributed setup: initializes ranks
src/ddp_gradient_compression/train.py - Training runner → Dataset and loaders: prepares loaders
src/ddp_gradient_compression/train.py - Training runner → Model builder: builds model
src/ddp_gradient_compression/train.py - Training runner → PyTorch DDP: wraps model
src/ddp_gradient_compression/train.py - Training runner → Hook selection: builds hook
src/ddp_gradient_compression/train.py - PyTorch DDP → Hook selection: invokes registered hook
README.md - Hook selection → Top-K hook: selects
src/ddp_gradient_compression/hooks/factory.py - Hook selection → PowerSGD hook: selects
src/ddp_gradient_compression/hooks/factory.py - Hook selection → Dense and FP16: selects
src/ddp_gradient_compression/hooks/factory.py - Top-K hook → Gloo collectives: gathers sparse payload
src/ddp_gradient_compression/hooks/topk.py - PowerSGD hook → Gloo collectives: reduces factors
src/ddp_gradient_compression/hooks/powersgd.py - Distributed setup → ACCL process group: registers optional backend
src/ddp_gradient_compression/distributed.py - ACCL process group → Rank-device config: loads rank mapping
src/ddp_gradient_compression/backends/accl.py - ACCL process group → PyACCL adapter: creates runtime
src/ddp_gradient_compression/backends/accl.py - PyACCL adapter → Rank-device config: uses configuration
src/ddp_gradient_compression/backends/accl_runtime.py - PyACCL adapter → ACCL device or emulator: runs collectives
src/ddp_gradient_compression/backends/accl_runtime.py - Training runner → Epoch evaluation: evaluates epochs
README.md - Training runner → CSV metrics: writes epoch metrics
src/ddp_gradient_compression/train.py