# girayonur/composite-gradient-learning architecture

> Architecture diagram of the GitHub repository girayonur/composite-gradient-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/girayonur/composite-gradient-learning
- Repository: https://github.com/girayonur/composite-gradient-learning

## Overview

This repository contains the paper’s composite-gradient learning experiments for traffic-network control. The main workflow trains an agent against a simulated traffic environment, where learned ramp-meter actions interact with an MPC controller and a METANET traffic model; saved agents are then evaluated and results recorded. The two network directories implement parallel versions for different network sizes. README capabilities also include demand scenarios, gradient diagnostics, and saved evaluation results. Relationships reflect the excerpts, SOURCE INDEX references, and README; unsampled details are not assumed.

Main entry points: [`network_1/src/traffic_env_dmpc.py`](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/traffic_env_dmpc.py) (Network 1 environment), [`network_2/src/traffic_env_dmpc.py`](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/traffic_env_dmpc.py) (Network 2 environment), [`network_1/src/train_ddpg_dmpc.py`](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/train_ddpg_dmpc.py) (Network 1 learner), [`network_2/src/train_ddpg_dmpc.py`](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/train_ddpg_dmpc.py) (Network 2 learner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiments["Experiment workflows"]
  node_n1_train["Network 1 training<br/>[train.py]"]
  node_n2_train["Network 2 training<br/>[train.py]"]
  node_n1_eval["Network 1 evaluation<br/>[evaluate.py]"]
  node_n2_eval["Network 2 evaluation<br/>[evaluate.py]"]
  node_n1_runner["Network 1 evaluator"]
  node_n2_runner["Network 2 evaluator"]
end

subgraph group_control["Traffic control"]
  node_n1_env["Network 1 environment"]
  node_n2_env["Network 2 environment"]
  node_n1_demand["Network 1 demands<br/>[demands.py]"]
  node_n2_demand["Network 2 demands<br/>[demands.py]"]
  node_n1_mpc["Network 1 MPC<br/>[dmpc.py]"]
  node_n2_mpc["Network 2 MPC<br/>[dmpc.py]"]
  node_n1_model["Network 1 traffic model<br/>[metanet_model.py]"]
  node_n2_model["Network 2 traffic model<br/>[metanet_model.py]"]
  node_n1_solver["Network 1 PI solver"]
  node_n2_solver["Network 2 PI solver"]
  node_n1_params["Network 1 parameters<br/>[params.py]"]
  node_n2_params["Network 2 parameters<br/>[params.py]"]
end

subgraph group_learning["Learning and diagnostics"]
  node_n1_trainer["Network 1 learner<br/>[train_ddpg_dmpc.py]"]
  node_n2_trainer["Network 2 learner<br/>[train_ddpg_dmpc.py]"]
  node_n1_metrics["Network 1 gradient metrics"]
  node_n2_metrics["Network 2 gradient metrics"]
end

subgraph group_artifacts["Experiment artifacts"]
  node_agents[("Trained agents")]
  node_results["Evaluation results"]
  node_gradients["Gradient logs"]
end

node_researcher(("Researcher"))

node_researcher -->|"starts training"| node_n1_train
node_researcher -->|"starts training"| node_n2_train
node_researcher -->|"starts evaluation"| node_n1_eval
node_researcher -->|"starts evaluation"| node_n2_eval
node_n1_train -->|"runs learner"| node_n1_trainer
node_n2_train -->|"runs learner"| node_n2_trainer
node_n1_eval -->|"runs evaluation"| node_n1_runner
node_n2_eval -->|"runs evaluation"| node_n2_runner
node_n1_trainer -->|"steps environment"| node_n1_env
node_n2_trainer -->|"steps environment"| node_n2_env
node_n1_trainer -->|"uses controller"| node_n1_mpc
node_n2_trainer -->|"uses controller"| node_n2_mpc
node_n1_trainer -->|"computes diagnostics"| node_n1_metrics
node_n2_trainer -->|"computes diagnostics"| node_n2_metrics
node_n1_env -->|"generates demand"| node_n1_demand
node_n2_env -->|"generates demand"| node_n2_demand
node_n1_env -->|"requests control"| node_n1_mpc
node_n2_env -->|"requests control"| node_n2_mpc
node_n1_env -->|"simulates traffic"| node_n1_model
node_n2_env -->|"simulates traffic"| node_n2_model
node_n1_mpc -->|"optimizes rollout"| node_n1_model
node_n2_mpc -->|"optimizes rollout"| node_n2_model
node_n1_mpc -->|"uses solver"| node_n1_solver
node_n2_mpc -->|"uses solver"| node_n2_solver
node_n1_env -->|"configures model"| node_n1_params
node_n2_env -->|"configures model"| node_n2_params
node_n1_runner -->|"evaluates policy"| node_n1_env
node_n2_runner -->|"evaluates policy"| node_n2_env
node_n1_trainer -.->|"saves models"| node_agents
node_n2_trainer -.->|"saves models"| node_agents
node_n1_runner -.->|"records metrics"| node_results
node_n2_runner -.->|"records metrics"| node_results
node_n1_metrics -.->|"logs gradients"| node_gradients
node_n2_metrics -.->|"logs gradients"| node_gradients

click node_n1_train "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/train.py"
click node_n2_train "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/train.py"
click node_n1_eval "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/evaluate.py"
click node_n2_eval "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/evaluate.py"
click node_n1_runner "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/evaluate_ddpg_dmpc_noise_seeds.py"
click node_n2_runner "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/evaluate_ddpg_dmpc_noise_seeds.py"
click node_n1_env "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/traffic_env_dmpc.py"
click node_n2_env "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/traffic_env_dmpc.py"
click node_n1_demand "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/demands.py"
click node_n2_demand "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/demands.py"
click node_n1_mpc "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/dmpc.py"
click node_n2_mpc "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/dmpc.py"
click node_n1_model "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/metanet_model.py"
click node_n2_model "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/metanet_model.py"
click node_n1_solver "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/mpc_solve_pi_alinea.py"
click node_n2_solver "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/mpc_solve_pi_alinea.py"
click node_n1_params "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/params.py"
click node_n2_params "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/params.py"
click node_n1_trainer "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/train_ddpg_dmpc.py"
click node_n2_trainer "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/train_ddpg_dmpc.py"
click node_n1_metrics "https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/gradient_metrics.py"
click node_n2_metrics "https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/gradient_metrics.py"
click node_agents "https://github.com/girayonur/composite-gradient-learning/tree/main/network_1/agents"
click node_results "https://github.com/girayonur/composite-gradient-learning/tree/main/network_1/results/paper"
click node_gradients "https://github.com/girayonur/composite-gradient-learning/tree/main/network_1/training/gradients"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n1_train,node_n2_train,node_n1_eval,node_n2_eval,node_n1_runner,node_n2_runner toneBlue
class node_n1_env,node_n2_env,node_n1_demand,node_n2_demand,node_n1_mpc,node_n2_mpc,node_n1_model,node_n2_model,node_n1_solver,node_n2_solver,node_n1_params,node_n2_params toneAmber
class node_n1_trainer,node_n2_trainer,node_n1_metrics,node_n2_metrics toneMint
class node_agents,node_results,node_gradients toneRose
class node_researcher toneIndigo
```

## Components

### Experiment workflows

- [Network 1 training](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/train.py): `network_1/train.py`
- [Network 2 training](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/train.py): `network_2/train.py`
- [Network 1 evaluation](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/evaluate.py): `network_1/evaluate.py`
- [Network 2 evaluation](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/evaluate.py): `network_2/evaluate.py`
- [Network 1 evaluator](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/evaluate_ddpg_dmpc_noise_seeds.py): `network_1/src/evaluate_ddpg_dmpc_noise_seeds.py`
- [Network 2 evaluator](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/evaluate_ddpg_dmpc_noise_seeds.py): `network_2/src/evaluate_ddpg_dmpc_noise_seeds.py`

### Traffic control

- [Network 1 environment](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/traffic_env_dmpc.py): `network_1/src/traffic_env_dmpc.py`
- [Network 2 environment](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/traffic_env_dmpc.py): `network_2/src/traffic_env_dmpc.py`
- [Network 1 demands](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/demands.py): `network_1/src/demands.py`
- [Network 2 demands](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/demands.py): `network_2/src/demands.py`
- [Network 1 MPC](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/dmpc.py): `network_1/src/dmpc.py`
- [Network 2 MPC](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/dmpc.py): `network_2/src/dmpc.py`
- [Network 1 traffic model](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/metanet_model.py): `network_1/src/metanet_model.py`
- [Network 2 traffic model](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/metanet_model.py): `network_2/src/metanet_model.py`
- [Network 1 PI solver](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/mpc_solve_pi_alinea.py): `network_1/src/mpc_solve_pi_alinea.py`
- [Network 2 PI solver](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/mpc_solve_pi_alinea.py): `network_2/src/mpc_solve_pi_alinea.py`
- [Network 1 parameters](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/params.py): `network_1/src/params.py`
- [Network 2 parameters](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/params.py): `network_2/src/params.py`

### Learning and diagnostics

- [Network 1 learner](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/train_ddpg_dmpc.py): `network_1/src/train_ddpg_dmpc.py`
- [Network 2 learner](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/train_ddpg_dmpc.py): `network_2/src/train_ddpg_dmpc.py`
- [Network 1 gradient metrics](https://github.com/girayonur/composite-gradient-learning/blob/main/network_1/src/gradient_metrics.py): `network_1/src/gradient_metrics.py`
- [Network 2 gradient metrics](https://github.com/girayonur/composite-gradient-learning/blob/main/network_2/src/gradient_metrics.py): `network_2/src/gradient_metrics.py`

### Experiment artifacts

- [Trained agents](https://github.com/girayonur/composite-gradient-learning/tree/main/network_1/agents): `network_1/agents`
- [Evaluation results](https://github.com/girayonur/composite-gradient-learning/tree/main/network_1/results/paper): `network_1/results/paper`
- [Gradient logs](https://github.com/girayonur/composite-gradient-learning/tree/main/network_1/training/gradients): `network_1/training/gradients`

### Other components

- **Researcher**

## Connections

- Researcher → Network 1 training: starts training (evidence: `README.md`)
- Researcher → Network 2 training: starts training (evidence: `README.md`)
- Researcher → Network 1 evaluation: starts evaluation (evidence: `README.md`)
- Researcher → Network 2 evaluation: starts evaluation (evidence: `README.md`)
- Network 1 training → Network 1 learner: runs learner (evidence: `README.md`)
- Network 2 training → Network 2 learner: runs learner (evidence: `README.md`)
- Network 1 evaluation → Network 1 evaluator: runs evaluation (evidence: `README.md`)
- Network 2 evaluation → Network 2 evaluator: runs evaluation (evidence: `README.md`)
- Network 1 learner → Network 1 environment: steps environment (evidence: `network_1/src/train_ddpg_dmpc.py`)
- Network 2 learner → Network 2 environment: steps environment (evidence: `network_2/src/train_ddpg_dmpc.py`)
- Network 1 learner → Network 1 MPC: uses controller (evidence: `network_1/src/train_ddpg_dmpc.py`)
- Network 2 learner → Network 2 MPC: uses controller (evidence: `network_2/src/train_ddpg_dmpc.py`)
- Network 1 learner → Network 1 gradient metrics: computes diagnostics (evidence: `network_1/src/train_ddpg_dmpc.py`)
- Network 2 learner → Network 2 gradient metrics: computes diagnostics (evidence: `network_2/src/train_ddpg_dmpc.py`)
- Network 1 environment → Network 1 demands: generates demand (evidence: `network_1/src/traffic_env_dmpc.py`)
- Network 2 environment → Network 2 demands: generates demand (evidence: `network_2/src/traffic_env_dmpc.py`)
- Network 1 environment → Network 1 MPC: requests control (evidence: `network_1/src/traffic_env_dmpc.py`)
- Network 2 environment → Network 2 MPC: requests control (evidence: `network_2/src/traffic_env_dmpc.py`)
- Network 1 environment → Network 1 traffic model: simulates traffic (evidence: `network_1/src/traffic_env_dmpc.py`)
- Network 2 environment → Network 2 traffic model: simulates traffic (evidence: `network_2/src/traffic_env_dmpc.py`)
- Network 1 MPC → Network 1 traffic model: optimizes rollout (evidence: `network_1/src/dmpc.py`)
- Network 2 MPC → Network 2 traffic model: optimizes rollout (evidence: `network_2/src/dmpc.py`)
- Network 1 MPC → Network 1 PI solver: uses solver (evidence: `network_1/src/dmpc.py`)
- Network 2 MPC → Network 2 PI solver: uses solver (evidence: `network_2/src/dmpc.py`)
- Network 1 environment → Network 1 parameters: configures model (evidence: `network_1/src/traffic_env_dmpc.py`)
- Network 2 environment → Network 2 parameters: configures model (evidence: `network_2/src/traffic_env_dmpc.py`)
- Network 1 evaluator → Network 1 environment: evaluates policy (evidence: `network_1/src/evaluate_ddpg_dmpc_noise_seeds.py`)
- Network 2 evaluator → Network 2 environment: evaluates policy (evidence: `network_2/src/evaluate_ddpg_dmpc_noise_seeds.py`)
- Network 1 learner → Trained agents: saves models (evidence: `README.md`)
- Network 2 learner → Trained agents: saves models (evidence: `README.md`)
- Network 1 evaluator → Evaluation results: records metrics (evidence: `README.md`)
- Network 2 evaluator → Evaluation results: records metrics (evidence: `README.md`)
- Network 1 gradient metrics → Gradient logs: logs gradients (evidence: `README.md`)
- Network 2 gradient metrics → Gradient logs: logs gradients (evidence: `README.md`)
