abeenoch/distributedtraining

How abeenoch/distributedtraining works

This repository is a distributed deep-learning training framework. Its main workflow configures a run, loads and shards data, then trains through either a parameter-server worker/RPC path or a DDP path; training produces checkpoints and metrics that support recovery and performance comparison. The README documents baseline training, sync/async aggregation, compression, fault tolerance, and reporting, so those branches are represented. Arrows are limited to relationships shown by excerpts, SOURCE INDEX references, or README statements; unsampled details are omitted.

Main entry points: worker.py Training Worker · run_final_checkpoint_validation.py Checkpoint Validation · parameter_server.py Parameter Server · metrics.py Metrics Logger

Components 20

Configuration and Data

Run Configuration src/config.py
Dataset Loading src/dataset.py

Training Architectures

Baseline Trainer src/trainer.py
DDP Trainer src/ddp_trainer.py
Baseline Runner run_baseline_training.py
DDP Runner run_ddp_training.py

Distributed Runtime

Parameter Server src/parameter_server.py
Training Worker src/worker.py
Fault Coordinator src/coordinator.py
RPC Services src/communication/services.py
Tensor Serialization src/communication/serialization.py
RPC Retry src/communication/retry.py
Gradient Compression src/compression.py
RPC Protocol src/communication/protos/distributed.proto
Parameter Server Runner run_parameter_server_training.py

Metrics and Reporting

Metrics Logger src/metrics.py
Performance Analysis src/performance.py
Comparison Reports src/reporting.py
Checkpoint Validation run_final_checkpoint_validation.py

Other components

Run Operator

Connections 40

Run Operator → Baseline Runner: starts README.md
Run Operator → DDP Runner: starts README.md
Run Operator → Parameter Server Runner: starts README.md
Run Operator → Checkpoint Validation: starts README.md
Baseline Runner → Run Configuration: loads run_baseline_training.py
Baseline Runner → Baseline Trainer: runs run_baseline_training.py
DDP Runner → Run Configuration: loads run_ddp_training.py
DDP Runner → Dataset Loading: loads run_ddp_training.py
DDP Runner → DDP Trainer: launches run_ddp_training.py
Parameter Server Runner → Run Configuration: loads run_parameter_server_training.py
Parameter Server Runner → Dataset Loading: loads run_parameter_server_training.py
Parameter Server Runner → Fault Coordinator: starts run_parameter_server_training.py
Parameter Server Runner → Parameter Server: starts run_parameter_server_training.py
Checkpoint Validation → Fault Coordinator: tests recovery run_final_checkpoint_validation.py
Checkpoint Validation → Parameter Server: validates run_final_checkpoint_validation.py
Checkpoint Validation → DDP Trainer: validates run_final_checkpoint_validation.py
Checkpoint Validation → Baseline Trainer: compares run_final_checkpoint_validation.py
Baseline Trainer → Run Configuration: uses src/trainer.py
Baseline Trainer → Dataset Loading: loads src/trainer.py
DDP Trainer → Run Configuration: uses src/ddp_trainer.py
DDP Trainer → Metrics Logger: logs src/ddp_trainer.py
Fault Coordinator → Training Worker: manages src/coordinator.py
Fault Coordinator → RPC Services: registers src/coordinator.py
Training Worker → Parameter Server: sends gradients src/worker.py
Training Worker → RPC Services: calls src/worker.py
Training Worker → Tensor Serialization: serializes tensors src/worker.py
Training Worker → Gradient Compression: compresses gradients src/worker.py
Training Worker → Dataset Loading: uses shards src/worker.py
Training Worker → Metrics Logger: reports src/worker.py
Parameter Server → RPC Services: implements src/parameter_server.py
Parameter Server → Tensor Serialization: decodes tensors src/parameter_server.py
Parameter Server → Gradient Compression: decompresses gradients src/parameter_server.py
RPC Services → RPC Protocol: uses messages src/communication/services.py
Tensor Serialization → RPC Protocol: encodes messages src/communication/serialization.py
Fault Coordinator → Metrics Logger: records metrics src/coordinator.py
Metrics Logger → Comparison Reports: feeds reports generate_performance_report.py
Metrics Logger → Performance Analysis: supports analysis
RPC Retry → RPC Services: retries calls
Checkpoint Validation → Performance Analysis: measures runs run_final_checkpoint_validation.py
Checkpoint Validation → Metrics Logger: collects metrics run_final_checkpoint_validation.py

Drawn by GitDiagram on November 13, 2026 from the repository's file tree, README and source files. Read as Markdown ·