How felisat/clustered-federated-learning works
Project Type and Purpose
• This is a research‐oriented Python simulation for “Clustered Federated Learning,” a specialized form of federated multi‐task optimization under privacy constraints.
• The goal is to simulate a server coordinating multiple devices (clients), each training local models on private data, then clustering similar tasks and aggregating updates.
File Structure
• clustered_federated_learning.ipynb: the central orchestration notebook that ties together data loading, device simulation, training loops, and clustering logic.
• data_utils.py: utilities for loading, partitioning, and preprocessing datasets for each simulated device.
• fl_devices.py: defines the “Device” abstraction—local training loops, update generation, and communication stubs.
• models.py: implements the machine‐learning model(s) used on each device (e.g., a neural network or logistic regression).
• helper.py: clustering routines and aggregation functions (e.g., distance computations, K-means on model updates).
• README.md / .gitignore: project metadata and environment exclusions.
Main System Components
a. Central Server (Orchestrator)
• Lives in the Jupyter notebook (clustered_federated_learning.ipynb)
• Coordinates rounds of communication, clustering, and model aggregation.
b. Device Simulation Layer
• fl_devices.py → Device classes simulate on‐device training and remote updates.
• Each Device holds its own data partition (via data_utils.py) and local model instance (from models.py).
c. Data Layer
• data_utils.py handles dataset ingestion and splits per device.
d. Model Layer
• models.py defines the architecture and forward/backward logic.
e. Clustering & Aggregation
• helper.py implements model‐update similarity measures and clustering (e.g., K-means) to group devices.
f. User/Research Interface
• The Jupyter notebook orchestrates configuration of hyperparameters, triggers training and clustering rounds, and visualizes results.
Relationships and Interactions
• The Orchestrator initializes a global model prototype.
• For each round:
1. The Orchestrator pushes the current global model to all Device instances.
2. Each Device loads its local data, trains for a fixed number of epochs, and returns a model update (gradient or weight delta).
3. helper.py clusters these updates into groups of similar tasks.
4. Within each cluster, updates are aggregated (e.g., averaged) to form a cluster‐specific model.
5. Optionally, cluster‐level models are merged back into a new global model or kept separate.
• Results are logged and visualized in the notebook.
Architectural Patterns and Design Principles
• Separation of Concerns: data loading, model definition, device logic, and clustering are modularized.
• Broker/Worker Pattern: the notebook acts as broker/orchestrator; devices are workers.
• Iterative Communication Rounds: fixed phases of “broadcast → local compute → collect → cluster & aggregate.”
Technology and Framework Callouts
• Core Language: Python 3.x
• Notebook Environment: Jupyter
• Numeric Libraries: NumPy, potentially PyTorch or TensorFlow if models.py uses them
• Clustering: scikit-learn or custom K-means routines in helper.py