How faresmalik/fesvibs works
Project type and purpose
• Type: Open-source research toolkit implementing federated split learning of a Vision Transformer (ViT) with block sampling.
• Purpose: Enable hospitals/clients to collaboratively train a ViT for medical image classification without sharing raw data, by splitting the model between client & server, sampling intermediate features, and federating server-side weights.
File structure
Top-level scripts:
• centralized.py – centralized (non-federated) end-to-end training/testing.
• local.py – purely local per-client training/testing.
• SLViT.py – vanilla split learning with ViT (with optional DP).
• FeSViBS.py – federated split learning with block sampling (with optional DP).
Core libraries:
• dataset.py – data loading & preprocessing for HAM10000, BloodMNIST, Fed-ISIC2019.
• models.py – defines ViT blocks, classification head, block sampling module.
• utils.py – training loops, logging, checkpointing, metrics.
Assets:
• Figures/method.PNG – high-level method illustration.
• requirements.txt, LICENSE, README.md
README for architecture clues
• Two main flows: Split-Learning (SLViT) and Federated Split-Learning (FeSViBS).
• Key components: client-side ViT block(s), server-side ViT block(s), block sampling & pseudo-token distillation, federated aggregation of server parameters.
Main components & interactions
a. Clients (N hospitals)
– Data Loader (dataset.py)
– Client-side ViT segment (initial_block…k)
– Local optimizer & backprop for client segment (utils.py)
– Transmits hidden activations and receives gradients / pseudo-tokens.
b. Server
– Server-side ViT segment (blocks k+1…end in models.py)
– Block-Sampling Module (sample patch tokens → pseudo class token)
– Classification Head → computes loss ← sends gradients back.
– Federated Aggregator (FedAvg) for server segment weights & sampler parameters
– Differential Privacy (optional) on gradients / weight updates
c. Data Stores & External Services
– On-disk datasets (HAM10000 folder, FLamby path)
– MedMNIST API (BloodMNIST)
d. Orchestrator Scripts
– FeSViBS.py coordinates per-round local split-training, block sampling, then federated aggregation.
– SLViT.py / centralized.py / local.py follow analogous flows without federation or splitting.
Architectural patterns & design principles
– Split Learning: partitioning a single neural network across clients & server.
– Federated Learning: aggregating server-side model parameters across clients.
– Modular design: clear separation of data I/O, model definition, training logic, and orchestration.
– Extensibility: support for DP, varying block split points, pluggable optimizers.