How sameernadaf21/survey-federated-learning works
Project Type and Purpose
• This is a Python-based federated-learning framework/experimental tool (“sameernadaf21survey-federated-learning”).
• It orchestrates multiple client processes that each train a local ML model on private data, and a central server process that aggregates client updates into a global model.
File-Structure Analysis
• Top-level directories:
• client/
• server/
• conf/ (Hydra configuration)
• outputs/ and client/outputs/, server/outputs/ (Hydra run outputs)
• client/ contains:
• rem_client.py (client entrypoint)
• client_class.py (encapsulates client logic)
• dataset.py (local data loader)
• model.py (client-side model definition)
• server/ contains:
• server.py (server entrypoint)
• server_helper.py (aggregation utilities)
• dataset.py (if the server simulates a global dataset or validation set)
• model.py (global model definition or aggregator interface)
• conf/base.yaml: global Hydra config (hyperparameters, addresses, schedules).
• Hydra outputs folders: store logs, config snapshots, and artifacts (results.pkl).
Key Architectural Patterns and Components
a. Components:
1) Config Manager (Hydra)
2) Client Nodes (1…N)
– Client Class
– Local Dataset
– Local Model
– rem_client.py (invokes training, sends updates)
3) Server Node
– Aggregation Engine (server.py + server_helper.py)
– Global Model
– (Optional) Global/validation dataset
4) Storage/Logging subsystem
– Hydra outputs folders for client and server runs
b. Relationships & Interactions:
– Clients read Hydra configs → train local models → serialize parameter updates/logs → send to Server
– Server listens for client updates → aggregates (e.g., FedAvg) → updates global model → writes new model and logs
– (Optional) Server distributes the new global model back to clients for next round
– All components read/write to the outputs directory under Hydra’s control
c. Technologies & Frameworks:
– Python 3.x
– Hydra for configuration management
– ML framework (e.g., PyTorch or TensorFlow—see requirements.txt)
– Standard libraries for file I/O, logging, serialization (pickle)
d. Architectural Patterns:
– Federated-learning pattern (client-server aggregation)
– Hydra-driven experiment orchestration
– Modular code separation: data-loading, modeling, orchestration
a. Sketch a Config Manager block at the top (Hydra).
– Hydra for config
– Storage block (server/outputs)
d. In the Server block, annotate the aggregation step (“FedAvg” or similar) and link to the “model.py” artifact.
e. In each Client block, annotate the training loop (“dataset.py” → “model.py” → “update send”).
• Clear component labels (e.g., “Client Trainer”, “Server Aggregator”, “Hydra Config”)