# mihirchouhan/data-center-federated-learning architecture

> Architecture diagram of the GitHub repository mihirchouhan/data-center-federated-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mihirchouhan/data-center-federated-learning
- Repository: https://github.com/mihirchouhan/data-center-federated-learning

## Overview

This is a resumable M100 forecasting and research-training template. The user-facing workflow downloads and verifies Dataset 5, prepares chronological forecasting windows, trains recurrent models, then evaluates held-out forecasts and writes artifacts. Alongside that baseline, the repository contains centralized and simulated FedAvg experiments, historical-coverage audits, and missing-data policy experiments. The graph includes shared data, model, evaluation, resource, and durable-I/O layers. Relationships beyond README-described stages are limited to references recorded in the source index; unsampled implementations are not assumed to have additional call paths.

Main entry points: [`src/federated_rounds.py`](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_rounds.py) (Round Orchestration), [`pipeline.py`](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/pipeline.py) (Pipeline CLI), [`src/experiment_eval.py`](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_eval.py) (Experiment Evaluation), [`src/history_data_v4.py`](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/history_data_v4.py) (Historical Data Access)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["M100 Workflow"]
  node_pipeline["Pipeline CLI<br/>[pipeline.py]"]
  node_download["Dataset Download<br/>[download.py]"]
  node_preprocess["Window Preparation<br/>[preprocess.py]"]
  node_main["Training CLI<br/>[main.py]"]
end

subgraph group_experiments["Experiment Framework"]
  node_experiments_cli["Experiment Runner<br/>[experiments.py]"]
  node_exp_data[("Frozen Experiment Data<br/>[experiment_data.py]")]
  node_exp_models["Experiment Models"]
  node_exp_eval["Experiment Evaluation<br/>[experiment_eval.py]"]
  node_resources["Resource Monitoring"]
  node_central_sgd["Centralized SGD<br/>[centralized_sgd.py]"]
  node_central_eval["Centralized Evaluation"]
end

subgraph group_federated["Federated Training"]
  node_federated_cli["FedAvg Runner<br/>[federated.py]"]
  node_fed_data["Federated Partitions<br/>[federated_data.py]"]
  node_fed_rounds["Round Orchestration"]
  node_fed_train["Client Updates<br/>[federated_train.py]"]
  node_fed_io["Round Commits<br/>[federated_io.py]"]
  node_fed_eval["FedAvg Evaluation"]
end

subgraph group_data_research["Data Research"]
  node_history_data["Historical Data Access<br/>[history_data_v4.py]"]
  node_history_coverage["Coverage Audit"]
  node_cohorts["Cohort Builder"]
  node_missing_train["Missing-Data Training"]
  node_policy_eval["Policy Evaluation"]
  node_training_data["Policy Training Data<br/>[training_data.py]"]
  node_training_io["Policy Run I/O<br/>[training_io.py]"]
end

subgraph group_shared["Shared Runtime"]
  node_model["Forecast Model<br/>[model.py]"]
  node_plot["Forecast Plotting<br/>[forecast_plot.py]"]
  node_io["Durable I/O<br/>[io_utils.py]"]
  node_multinode["Multi-node Data<br/>[multinode.py]"]
end

node_user(("Researcher"))

node_user -->|"runs"| node_pipeline
node_pipeline -->|"downloads data"| node_download
node_pipeline -->|"prepares data"| node_preprocess
node_pipeline -->|"trains"| node_main
node_pipeline -->|"plots forecasts"| node_plot
node_pipeline -->|"uses"| node_io
node_main -->|"uses"| node_model
node_main -->|"uses"| node_io
node_pipeline -->|"uses"| node_model
node_experiments_cli -->|"uses"| node_exp_data
node_experiments_cli -->|"uses"| node_exp_models
node_experiments_cli -->|"uses"| node_exp_eval
node_experiments_cli -->|"uses"| node_resources
node_exp_models -->|"uses"| node_exp_data
node_exp_models -->|"uses"| node_model
node_exp_eval -->|"reads"| node_exp_data
node_exp_eval -->|"plots"| node_plot
node_exp_eval -->|"writes"| node_io
node_federated_cli -->|"uses"| node_exp_data
node_federated_cli -->|"runs rounds"| node_fed_rounds
node_fed_rounds -->|"uses"| node_fed_data
node_fed_rounds -->|"uses"| node_fed_train
node_fed_rounds -->|"commits"| node_fed_io
node_fed_rounds -->|"uses"| node_exp_eval
node_fed_rounds -->|"monitors"| node_resources
node_fed_rounds -->|"initializes"| node_model
node_fed_eval -->|"reads"| node_fed_io
node_fed_eval -->|"uses"| node_fed_rounds
node_central_sgd -->|"uses"| node_fed_data
node_central_sgd -->|"uses"| node_fed_io
node_central_sgd -->|"uses"| node_fed_rounds
node_central_eval -->|"uses"| node_fed_eval
node_history_coverage -->|"uses"| node_history_data
node_history_data -->|"uses"| node_multinode
node_history_data -->|"uses"| node_preprocess
node_cohorts -->|"uses"| node_history_coverage
node_cohorts -->|"uses"| node_history_data
node_missing_train -->|"uses"| node_training_data
node_missing_train -->|"uses"| node_training_io
node_missing_train -->|"scores with"| node_policy_eval
node_policy_eval -->|"uses"| node_training_data
node_policy_eval -->|"uses"| node_training_io
node_training_data -->|"uses"| node_history_data
node_training_io -->|"uses"| node_io

click node_pipeline "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/pipeline.py"
click node_download "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/download.py"
click node_preprocess "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/preprocess.py"
click node_main "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/main.py"
click node_model "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/model.py"
click node_plot "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/forecast_plot.py"
click node_io "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/io_utils.py"
click node_experiments_cli "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/experiments.py"
click node_exp_data "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_data.py"
click node_exp_models "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_models.py"
click node_exp_eval "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_eval.py"
click node_resources "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_resources.py"
click node_federated_cli "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/federated.py"
click node_fed_data "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_data.py"
click node_fed_rounds "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_rounds.py"
click node_fed_train "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_train.py"
click node_fed_io "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_io.py"
click node_fed_eval "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_evaluate.py"
click node_central_sgd "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/centralized_sgd.py"
click node_central_eval "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/centralized_evaluate.py"
click node_history_data "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/history_data_v4.py"
click node_history_coverage "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/history_coverage_v4.py"
click node_cohorts "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/build_training_cohorts.py"
click node_missing_train "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/run_policy_experiments.py"
click node_policy_eval "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/evaluate_policy_models.py"
click node_training_data "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/training_data.py"
click node_training_io "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/training_io.py"
click node_multinode "https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/multinode.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_download,node_preprocess,node_main toneBlue
class node_experiments_cli,node_exp_data,node_exp_models,node_exp_eval,node_resources,node_central_sgd,node_central_eval toneAmber
class node_federated_cli,node_fed_data,node_fed_rounds,node_fed_train,node_fed_io,node_fed_eval toneMint
class node_history_data,node_history_coverage,node_cohorts,node_missing_train,node_policy_eval,node_training_data,node_training_io toneRose
class node_model,node_plot,node_io,node_multinode,node_user toneIndigo
```

## Components

### M100 Workflow

- [Pipeline CLI](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/pipeline.py): `pipeline.py`
- [Dataset Download](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/download.py): `src/download.py`
- [Window Preparation](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/preprocess.py): `src/preprocess.py`
- [Training CLI](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/main.py): `main.py`

### Experiment Framework

- [Experiment Runner](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/experiments.py): `experiments.py`
- [Frozen Experiment Data](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_data.py): `src/experiment_data.py`
- [Experiment Models](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_models.py): `src/experiment_models.py`
- [Experiment Evaluation](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_eval.py): `src/experiment_eval.py`
- [Resource Monitoring](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/experiment_resources.py): `src/experiment_resources.py`
- [Centralized SGD](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/centralized_sgd.py): `src/centralized_sgd.py`
- [Centralized Evaluation](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/centralized_evaluate.py): `src/centralized_evaluate.py`

### Federated Training

- [FedAvg Runner](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/federated.py): `federated.py`
- [Federated Partitions](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_data.py): `src/federated_data.py`
- [Round Orchestration](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_rounds.py): `src/federated_rounds.py`
- [Client Updates](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_train.py): `src/federated_train.py`
- [Round Commits](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_io.py): `src/federated_io.py`
- [FedAvg Evaluation](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/federated_evaluate.py): `src/federated_evaluate.py`

### Data Research

- [Historical Data Access](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/history_data_v4.py): `src/history_data_v4.py`
- [Coverage Audit](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/history_coverage_v4.py): `src/history_coverage_v4.py`
- [Cohort Builder](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/build_training_cohorts.py): `src/v5_missing_data/build_training_cohorts.py`
- [Missing-Data Training](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/run_policy_experiments.py): `src/v5_missing_data/run_policy_experiments.py`
- [Policy Evaluation](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/evaluate_policy_models.py): `src/v5_missing_data/evaluate_policy_models.py`
- [Policy Training Data](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/training_data.py): `src/v5_missing_data/training_data.py`
- [Policy Run I/O](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/v5_missing_data/training_io.py): `src/v5_missing_data/training_io.py`

### Shared Runtime

- [Forecast Model](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/model.py): `src/model.py`
- [Forecast Plotting](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/forecast_plot.py): `src/forecast_plot.py`
- [Durable I/O](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/io_utils.py): `src/io_utils.py`
- [Multi-node Data](https://github.com/mihirchouhan/data-center-federated-learning/blob/main/src/multinode.py): `src/multinode.py`

### Other components

- **Researcher**

## Connections

- Researcher → Pipeline CLI: runs (evidence: `README.md`)
- Pipeline CLI → Dataset Download: downloads data (evidence: `README.md`)
- Pipeline CLI → Window Preparation: prepares data (evidence: `README.md`)
- Pipeline CLI → Training CLI: trains (evidence: `README.md`)
- Pipeline CLI → Forecast Plotting: plots forecasts (evidence: `pipeline.py`)
- Pipeline CLI → Durable I/O: uses (evidence: `pipeline.py`)
- Training CLI → Forecast Model: uses (evidence: `main.py`)
- Training CLI → Durable I/O: uses (evidence: `main.py`)
- Pipeline CLI → Forecast Model: uses (evidence: `pipeline.py`)
- Experiment Runner → Frozen Experiment Data: uses (evidence: `experiments.py`)
- Experiment Runner → Experiment Models: uses (evidence: `experiments.py`)
- Experiment Runner → Experiment Evaluation: uses (evidence: `experiments.py`)
- Experiment Runner → Resource Monitoring: uses (evidence: `experiments.py`)
- Experiment Models → Frozen Experiment Data: uses (evidence: `src/experiment_models.py`)
- Experiment Models → Forecast Model: uses (evidence: `src/experiment_models.py`)
- Experiment Evaluation → Frozen Experiment Data: reads (evidence: `src/experiment_eval.py`)
- Experiment Evaluation → Forecast Plotting: plots (evidence: `src/experiment_eval.py`)
- Experiment Evaluation → Durable I/O: writes (evidence: `src/experiment_eval.py`)
- FedAvg Runner → Frozen Experiment Data: uses (evidence: `federated.py`)
- FedAvg Runner → Round Orchestration: runs rounds (evidence: `federated.py`)
- Round Orchestration → Federated Partitions: uses (evidence: `src/federated_rounds.py`)
- Round Orchestration → Client Updates: uses (evidence: `src/federated_rounds.py`)
- Round Orchestration → Round Commits: commits (evidence: `src/federated_rounds.py`)
- Round Orchestration → Experiment Evaluation: uses (evidence: `src/federated_rounds.py`)
- Round Orchestration → Resource Monitoring: monitors (evidence: `src/federated_rounds.py`)
- Round Orchestration → Forecast Model: initializes (evidence: `src/federated_rounds.py`)
- FedAvg Evaluation → Round Commits: reads (evidence: `src/federated_evaluate.py`)
- FedAvg Evaluation → Round Orchestration: uses (evidence: `src/federated_evaluate.py`)
- Centralized SGD → Federated Partitions: uses (evidence: `src/centralized_sgd.py`)
- Centralized SGD → Round Commits: uses (evidence: `src/centralized_sgd.py`)
- Centralized SGD → Round Orchestration: uses (evidence: `src/centralized_sgd.py`)
- Centralized Evaluation → FedAvg Evaluation: uses (evidence: `src/centralized_evaluate.py`)
- Coverage Audit → Historical Data Access: uses (evidence: `src/history_coverage_v4.py`)
- Historical Data Access → Multi-node Data: uses (evidence: `src/history_data_v4.py`)
- Historical Data Access → Window Preparation: uses (evidence: `src/history_data_v4.py`)
- Cohort Builder → Coverage Audit: uses (evidence: `src/v5_missing_data/build_training_cohorts.py`)
- Cohort Builder → Historical Data Access: uses (evidence: `src/v5_missing_data/build_training_cohorts.py`)
- Missing-Data Training → Policy Training Data: uses (evidence: `src/v5_missing_data/run_policy_experiments.py`)
- Missing-Data Training → Policy Run I/O: uses (evidence: `src/v5_missing_data/run_policy_experiments.py`)
- Missing-Data Training → Policy Evaluation: scores with (evidence: `src/v5_missing_data/run_policy_experiments.py`)
- Policy Evaluation → Policy Training Data: uses (evidence: `src/v5_missing_data/evaluate_policy_models.py`)
- Policy Evaluation → Policy Run I/O: uses (evidence: `src/v5_missing_data/evaluate_policy_models.py`)
- Policy Training Data → Historical Data Access: uses (evidence: `src/v5_missing_data/training_data.py`)
- Policy Run I/O → Durable I/O: uses (evidence: `src/v5_missing_data/training_io.py`)