How epfml/powersgd works
PowerSGD is both a reusable distributed-gradient compression library and a paper-experiment codebase. The main user workflow is to compute model gradients, aggregate them across workers with low-rank PowerSGD compression, then apply the optimizer step. The research branch runs configured training and evaluation experiments across image-classification and language-modeling tasks, with multiple reducers and experiment schedules. The graph reflects the README and sampled files; some research-code wiring is documented by the tree and README but not confirmed in the excerpts.
Main entry points: train.py Experiment trainer · __init__.py Optimizer step · powersgd.py PowerSGD aggregator · utils.py Distributed utilities
Components 15
PowerSGD API
- Optimizer step
powersgd/__init__.py
Compression Runtime
- PowerSGD aggregator
powersgd/powersgd.py - Orthogonalization
powersgd/orthogonalization.py - Distributed utilities
powersgd/utils.py
Research Training
- Experiment trainer
paper-code/train.py - Gradient reducers
paper-code/gradient_reducers.py - Metric accumulator
paper-code/mean_accumulator.py - Cluster management
paper-code/environment/cluster.py
Tasks And Experiments
- CIFAR task
paper-code/tasks/cifar.py - CIFAR models
paper-code/tasks/cifar_architectures - Language modeling
paper-code/tasks/language_modeling - Experiment schedules
paper-code/schedule
Other components
- Training caller
- Distributed workers
- Google Cloud
Connections 13
- Training caller → Optimizer step: calls
- Optimizer step → PowerSGD aggregator: aggregates gradients
- PowerSGD aggregator → Orthogonalization: orthogonalizes factors
- PowerSGD aggregator → Distributed utilities: all-reduces factors
- Distributed utilities → Distributed workers: communicates
- Optimizer step → Training caller: steps optimizer
- Experiment trainer → CIFAR task: trains or evaluates
- Experiment trainer → Language modeling: trains or evaluates
- CIFAR task → CIFAR models: builds model
- Experiment trainer → Gradient reducers: uses configured reducer
- Experiment trainer → Metric accumulator: aggregates metrics
- Experiment schedules → Experiment trainer: configures experiments
- Cluster management → Google Cloud: manages instances