# shuangqing-xu/clover architecture

> Architecture diagram of the GitHub repository shuangqing-xu/clover, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shuangqing-xu/clover
- Repository: https://github.com/shuangqing-xu/clover

## Overview

Clover is an academic prototype for federated model training with differential privacy and client-side sparsification, alongside a separate maliciously secure shuffle and distributed-DP utilities. The README documents the MNIST, FashionMNIST, and CIFAR-10 training workflow and a standalone shuffle run. Source excerpts show client training and masking, model search, and DP-query construction; the main orchestration and several subsystems are unsampled, so their internal wiring is represented conservatively.

Main entry points: [`Clover_api/Clover/Clover_api.py`](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/Clover_api.py) (Federated coordinator), [`MaliciousSecurity/distributed_dp/ddpquery_utils.py`](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/ddpquery_utils.py) (DP query factory), [`Clover_api/Clover/model_trainer.py`](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/model_trainer.py) (Model trainer), [`Clover-main.py`](https://github.com/shuangqing-xu/clover/blob/master/Clover-main.py) (Training entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_federated["Federated training"]
  node_clover_main["Training entry<br/>[Clover-main.py]"]
  node_clover_api["Federated coordinator<br/>[Clover_api.py]"]
  node_loaders["Dataset loaders"]
  node_client["Client training<br/>[client.py]"]
  node_sparse_client["Sparse client training<br/>[set_client.py]"]
  node_trainer["Model trainer<br/>[model_trainer.py]"]
  node_fedavg["FedAvg aggregation<br/>[fedavg_trainer.py]"]
end

subgraph group_models["Models and search"]
  node_models["Vision models"]
  node_darts_search["DARTS search<br/>[train_search.py]"]
  node_darts_network["Search network<br/>[model_search.py]"]
end

subgraph group_privacy["Privacy mechanisms"]
  node_dp_run["Distributed DP run<br/>[fl_run.py]"]
  node_dme_run["DME run<br/>[dme_run.py]"]
  node_dp_utils["DP orchestration<br/>[fl_utils.py]"]
  node_dp_query_factory["DP query factory<br/>[ddpquery_utils.py]"]
  node_compression["Compression query"]
  node_noise_queries["Noise queries"]
  node_accounting["Privacy accounting"]
end

subgraph group_secure["Secure shuffle"]
  node_shuffle_entry["Secure shuffle run"]
end

node_operator(("Experiment operator"))
node_participants(("Shuffle participants"))

node_operator -->|"starts training"| node_clover_main
node_clover_main -->|"runs federated training"| node_clover_api
node_clover_api -->|"loads datasets"| node_loaders
node_clover_api -->|"dispatches local training"| node_client
node_clover_api -.->|"dispatches sparse training"| node_sparse_client
node_client -->|"trains and tests"| node_trainer
node_sparse_client -->|"trains and tests"| node_trainer
node_trainer -->|"uses model"| node_models
node_clover_api -->|"coordinates aggregation"| node_fedavg
node_operator -->|"starts search"| node_darts_search
node_darts_search -->|"trains search network"| node_darts_network
node_operator -->|"starts DP run"| node_dp_run
node_operator -->|"starts DME run"| node_dme_run
node_dp_run -->|"uses training utilities"| node_dp_utils
node_dp_utils -->|"builds DP query"| node_dp_query_factory
node_dp_query_factory -->|"wraps query"| node_compression
node_dp_query_factory -->|"selects mechanism"| node_noise_queries
node_dp_query_factory -->|"calculates bounds"| node_accounting
node_participants -->|"runs shuffle protocol"| node_shuffle_entry

click node_clover_main "https://github.com/shuangqing-xu/clover/blob/master/Clover-main.py"
click node_clover_api "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/Clover_api.py"
click node_loaders "https://github.com/shuangqing-xu/clover/tree/master/Clover_api/data_preprocessing"
click node_client "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/client.py"
click node_sparse_client "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/set_client.py"
click node_trainer "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/model_trainer.py"
click node_fedavg "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/fedavg_trainer.py"
click node_models "https://github.com/shuangqing-xu/clover/tree/master/Clover_api/model/cv"
click node_darts_search "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/model/cv/darts/train_search.py"
click node_darts_network "https://github.com/shuangqing-xu/clover/blob/master/Clover_api/model/cv/darts/model_search.py"
click node_dp_run "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/fl_run.py"
click node_dme_run "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/dme_run.py"
click node_dp_utils "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/fl_utils.py"
click node_dp_query_factory "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/ddpquery_utils.py"
click node_compression "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/compression_query.py"
click node_noise_queries "https://github.com/shuangqing-xu/clover/tree/master/MaliciousSecurity/distributed_dp"
click node_accounting "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/accounting_utils.py"
click node_shuffle_entry "https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/rss_shuffle_mali.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_clover_main,node_clover_api,node_loaders,node_client,node_sparse_client,node_trainer,node_fedavg toneBlue
class node_models,node_darts_search,node_darts_network toneAmber
class node_dp_run,node_dme_run,node_dp_utils,node_dp_query_factory,node_compression,node_noise_queries,node_accounting toneMint
class node_shuffle_entry toneRose
class node_operator,node_participants toneIndigo
```

## Components

### Federated training

- [Training entry](https://github.com/shuangqing-xu/clover/blob/master/Clover-main.py): `Clover-main.py`
- [Federated coordinator](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/Clover_api.py): `Clover_api/Clover/Clover_api.py`
- [Dataset loaders](https://github.com/shuangqing-xu/clover/tree/master/Clover_api/data_preprocessing): `Clover_api/data_preprocessing`
- [Client training](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/client.py): `Clover_api/Clover/client.py`
- [Sparse client training](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/set_client.py): `Clover_api/Clover/set_client.py`
- [Model trainer](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/model_trainer.py): `Clover_api/Clover/model_trainer.py`
- [FedAvg aggregation](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/Clover/fedavg_trainer.py): `Clover_api/Clover/fedavg_trainer.py`

### Models and search

- [Vision models](https://github.com/shuangqing-xu/clover/tree/master/Clover_api/model/cv): `Clover_api/model/cv`
- [DARTS search](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/model/cv/darts/train_search.py): `Clover_api/model/cv/darts/train_search.py`
- [Search network](https://github.com/shuangqing-xu/clover/blob/master/Clover_api/model/cv/darts/model_search.py): `Clover_api/model/cv/darts/model_search.py`

### Privacy mechanisms

- [Distributed DP run](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/fl_run.py): `MaliciousSecurity/distributed_dp/fl_run.py`
- [DME run](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/dme_run.py): `MaliciousSecurity/distributed_dp/dme_run.py`
- [DP orchestration](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/fl_utils.py): `MaliciousSecurity/distributed_dp/fl_utils.py`
- [DP query factory](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/ddpquery_utils.py): `MaliciousSecurity/distributed_dp/ddpquery_utils.py`
- [Compression query](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/compression_query.py): `MaliciousSecurity/distributed_dp/compression_query.py`
- [Noise queries](https://github.com/shuangqing-xu/clover/tree/master/MaliciousSecurity/distributed_dp): `MaliciousSecurity/distributed_dp`
- [Privacy accounting](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/distributed_dp/accounting_utils.py): `MaliciousSecurity/distributed_dp/accounting_utils.py`

### Secure shuffle

- [Secure shuffle run](https://github.com/shuangqing-xu/clover/blob/master/MaliciousSecurity/rss_shuffle_mali.py): `MaliciousSecurity/rss_shuffle_mali.py`

### Other components

- **Experiment operator**
- **Shuffle participants**

## Connections

- Experiment operator → Training entry: starts training
- Training entry → Federated coordinator: runs federated training
- Federated coordinator → Dataset loaders: loads datasets
- Federated coordinator → Client training: dispatches local training
- Federated coordinator → Sparse client training: dispatches sparse training
- Client training → Model trainer: trains and tests
- Sparse client training → Model trainer: trains and tests
- Model trainer → Vision models: uses model
- Federated coordinator → FedAvg aggregation: coordinates aggregation
- Experiment operator → DARTS search: starts search
- DARTS search → Search network: trains search network
- Experiment operator → Distributed DP run: starts DP run
- Experiment operator → DME run: starts DME run
- Distributed DP run → DP orchestration: uses training utilities
- DP orchestration → DP query factory: builds DP query
- DP query factory → Compression query: wraps query
- DP query factory → Noise queries: selects mechanism
- DP query factory → Privacy accounting: calculates bounds
- Shuffle participants → Secure shuffle run: runs shuffle protocol