How wizard1203/gossipfl works
Main Components
a. Experiment Launcher & Config Manager
b. Data Preprocessing & Loaders
c. Model Definitions
d. Algorithm Modules
e. Communication Core (fedml_core)
f. Optimizers & Compression
g. Trainers & Aggregators
h. Logging, Metrics & Checkpointing
i. External Infrastructure (MPI/MQTT broker, GPUs, Machines)
Relationships & Data Flows
– The Experiment Launcher reads configuration files (from experiments/configs_*) and initializes the Topology Manager in the Communication Core.
– For each client and/or server process:
• It invokes the corresponding Algorithm API (e.g. CHOCO_SGD_API.py, FedAvgAPI.py, etc.).
• The API sets up its Trainer (in MyModelTrainer.py or model_trainer.py) with a Model (from model/…) and Optimizer (from optim/…).
• It pulls local data via the Data Preprocessing module (in data_preprocessing/… or data/…).
– At each training step, the Trainer computes gradients, calls the Communication Module (in fedml_core/distributed/communication) to exchange messages according to the Topology Manager (in fedml_core/distributed/topology).
– Optionally, Compression (in compression/…) is applied before sending updates.
– Updates arrive at other nodes or the central aggregator (in basePS/ps_aggregator.py or decentralized managers), which merge them and push new parameters back out.
– Meanwhile, Logging & Metrics (in utils/logger.py, utils/metrics.py, utils/checkpoint.py, utils/wandb_util.py) record progress and save checkpoints.
Architectural Patterns
– Plugin Architecture: Each algorithm lives in its own folder under algorithms/…, sharing a common API and message definitions.
– Separation of Concerns:
• Data concerns live under data_preprocessing/… and data/….
• Model architectures under model/….
• Low-level communication under fedml_core/….
• Experiment orchestration in experiments/… and configs/….
– Distributed Topology Abstraction: Centralized PS vs. fully decentralized via interchangeable topology managers.
– Communication Abstraction: Support for multiple backends – MPI, MQTT, etc., all conforming to base_com_manager.
Detailed Elements to Capture
– Experiment Launcher reads main_args.conf → kicks off MPI-based job (experiments/mpi_based/main.py).
– Communication Stack: fedml_core/distributed/communication/{mpi,mqtt} + fedml_core/distributed/communication/base_com_manager.py.
– Topology Management: Symmetric vs. Asymmetric (symmetric_topology_manager.py vs. asymmetric_topology_manager.py).
– Algorithm Flow: API → Trainer → Model → Optimizer → Communicator → Aggregator/Peer.
– Data Pipeline: Raw datasets (data/…) → Preprocessor (data_preprocessing/…) → IID/non-IID loaders → Trainer input.
– Metrics & Logging: utils/logger.py, utils/metrics.py, utils/checkpoint.py.
– Clearly separates the major layers (experiment orchestration, communication, algorithm logic, data, models).
– Shows how control and data move through the system during a federated training run.
– Highlights the project’s extensible plugin architecture and multi-backend communication support.