# microsoft/smart architecture

> Architecture diagram of the GitHub repository microsoft/smart, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/smart
- Repository: https://github.com/microsoft/smart

## Overview

SMART is an offline reinforcement-learning research system: it trains a multi-task control transformer on DeepMind Control replay data, then fine-tunes a policy for a selected downstream task. The graph follows the documented pretraining and downstream workflows and includes the replay/data, transformer, and environment layers. Some configuration-driven class selection is not visible in the excerpts, so those links are documented from the README rather than inferred from imports.

Main entry points: [`src/dmc_downstream.py`](https://github.com/microsoft/smart/blob/main/src/dmc_downstream.py) (Downstream CLI), [`src/dmc_pretrain.py`](https://github.com/microsoft/smart/blob/main/src/dmc_pretrain.py) (Pretrain CLI), [`src/datamodules/dmc_datamodule.py`](https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_datamodule.py) (DMC data modules), [`src/models/multitask_ct_module.py`](https://github.com/microsoft/smart/blob/main/src/models/multitask_ct_module.py) (Multi-task transformer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Training workflows"]
  node_pretrain["Pretrain CLI<br/>[dmc_pretrain.py]"]
  node_downstream["Downstream CLI<br/>[dmc_downstream.py]"]
  node_trainer["Lightning trainer<br/>[pl_utils.py]"]
  node_pretrain_cfg["Pretrain config<br/>[pretrain.yaml]"]
  node_downstream_cfg["Downstream config<br/>[downstream.yaml]"]
end

subgraph group_data["Offline data"]
  node_source_data["Replay datasets<br/>[dmc_datamodule.py]"]
  node_dmc_data["DMC data modules<br/>[dmc_datamodule.py]"]
  node_replay_buffer[("Replay buffer")]
  node_dummy_data["Test dataset"]
end

subgraph group_models["Control models"]
  node_multitask_model["Multi-task transformer"]
  node_control_model["Control transformer<br/>[ct_module.py]"]
  node_gpt["GPT backbone<br/>[mingpt.py]"]
  node_model_utils["Model utilities<br/>[utils.py]"]
end

subgraph group_environment["Environment runtime"]
  node_dmc_factory["DMC environment factory<br/>[__init__.py]"]
  node_dmc_wrapper["Gym wrapper<br/>[wrappers.py]"]
  node_dmc_suite["DeepMind Control"]
end

node_user(("Researcher"))
node_azure[("Azure data storage")]

node_user -->|"runs pretraining"| node_pretrain
node_user -->|"runs fine-tuning"| node_downstream
node_pretrain_cfg -->|"configures"| node_pretrain
node_downstream_cfg -->|"configures"| node_downstream
node_pretrain -->|"creates trainer"| node_trainer
node_downstream -->|"creates trainer"| node_trainer
node_pretrain -->|"instantiates model"| node_multitask_model
node_pretrain -->|"tests with"| node_dummy_data
node_downstream -->|"creates data module"| node_dmc_data
node_downstream -->|"tests with"| node_dummy_data
node_source_data -->|"downloaded from"| node_azure
node_dmc_data -->|"uses buffer"| node_replay_buffer
node_dmc_data -->|"creates environments"| node_dmc_factory
node_dmc_factory -->|"registers wrapper"| node_dmc_wrapper
node_dmc_wrapper -->|"loads tasks"| node_dmc_suite
node_multitask_model -->|"uses backbone"| node_gpt
node_control_model -->|"uses backbone"| node_gpt
node_multitask_model -->|"gets action sizes"| node_model_utils
node_control_model -->|"gets action sizes"| node_model_utils
node_dmc_data -->|"uses action helpers"| node_model_utils
node_downstream -.->|"selects configured model"| node_control_model
node_multitask_model -.->|"evaluates policy"| node_dmc_factory

click node_pretrain "https://github.com/microsoft/smart/blob/main/src/dmc_pretrain.py"
click node_downstream "https://github.com/microsoft/smart/blob/main/src/dmc_downstream.py"
click node_trainer "https://github.com/microsoft/smart/blob/main/src/utils/pl_utils.py"
click node_pretrain_cfg "https://github.com/microsoft/smart/blob/main/configs/pretrain.yaml"
click node_downstream_cfg "https://github.com/microsoft/smart/blob/main/configs/downstream.yaml"
click node_source_data "https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_datamodule.py"
click node_dmc_data "https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_datamodule.py"
click node_replay_buffer "https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_replay_buffer.py"
click node_dummy_data "https://github.com/microsoft/smart/blob/main/src/datamodules/dummy_datamodule.py"
click node_multitask_model "https://github.com/microsoft/smart/blob/main/src/models/multitask_ct_module.py"
click node_control_model "https://github.com/microsoft/smart/blob/main/src/models/ct_module.py"
click node_gpt "https://github.com/microsoft/smart/blob/main/src/models/components/mingpt.py"
click node_model_utils "https://github.com/microsoft/smart/blob/main/src/models/utils.py"
click node_dmc_factory "https://github.com/microsoft/smart/blob/main/src/envs/dmc2gym/dmc2gym/__init__.py"
click node_dmc_wrapper "https://github.com/microsoft/smart/blob/main/src/envs/dmc2gym/dmc2gym/wrappers.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pretrain,node_downstream,node_trainer,node_pretrain_cfg,node_downstream_cfg toneBlue
class node_source_data,node_dmc_data,node_replay_buffer,node_dummy_data,node_azure toneAmber
class node_multitask_model,node_control_model,node_gpt,node_model_utils toneMint
class node_dmc_factory,node_dmc_wrapper,node_dmc_suite toneRose
class node_user toneIndigo
```

## Components

### Training workflows

- [Pretrain CLI](https://github.com/microsoft/smart/blob/main/src/dmc_pretrain.py): `src/dmc_pretrain.py`
- [Downstream CLI](https://github.com/microsoft/smart/blob/main/src/dmc_downstream.py): `src/dmc_downstream.py`
- [Lightning trainer](https://github.com/microsoft/smart/blob/main/src/utils/pl_utils.py): `src/utils/pl_utils.py`
- [Pretrain config](https://github.com/microsoft/smart/blob/main/configs/pretrain.yaml): `configs/pretrain.yaml`
- [Downstream config](https://github.com/microsoft/smart/blob/main/configs/downstream.yaml): `configs/downstream.yaml`

### Offline data

- [Replay datasets](https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_datamodule.py): `src/datamodules/dmc_datamodule.py`
- [DMC data modules](https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_datamodule.py): `src/datamodules/dmc_datamodule.py`
- [Replay buffer](https://github.com/microsoft/smart/blob/main/src/datamodules/dmc_replay_buffer.py): `src/datamodules/dmc_replay_buffer.py`
- [Test dataset](https://github.com/microsoft/smart/blob/main/src/datamodules/dummy_datamodule.py): `src/datamodules/dummy_datamodule.py`

### Control models

- [Multi-task transformer](https://github.com/microsoft/smart/blob/main/src/models/multitask_ct_module.py): `src/models/multitask_ct_module.py`
- [Control transformer](https://github.com/microsoft/smart/blob/main/src/models/ct_module.py): `src/models/ct_module.py`
- [GPT backbone](https://github.com/microsoft/smart/blob/main/src/models/components/mingpt.py): `src/models/components/mingpt.py`
- [Model utilities](https://github.com/microsoft/smart/blob/main/src/models/utils.py): `src/models/utils.py`

### Environment runtime

- [DMC environment factory](https://github.com/microsoft/smart/blob/main/src/envs/dmc2gym/dmc2gym/__init__.py): `src/envs/dmc2gym/dmc2gym/__init__.py`
- [Gym wrapper](https://github.com/microsoft/smart/blob/main/src/envs/dmc2gym/dmc2gym/wrappers.py): `src/envs/dmc2gym/dmc2gym/wrappers.py`
- **DeepMind Control**

### Other components

- **Researcher**
- **Azure data storage**

## Connections

- Researcher → Pretrain CLI: runs pretraining (evidence: `README.md`)
- Researcher → Downstream CLI: runs fine-tuning (evidence: `README.md`)
- Pretrain config → Pretrain CLI: configures (evidence: `README.md`)
- Downstream config → Downstream CLI: configures (evidence: `README.md`)
- Pretrain CLI → Lightning trainer: creates trainer (evidence: `src/dmc_pretrain.py`)
- Downstream CLI → Lightning trainer: creates trainer (evidence: `src/dmc_downstream.py`)
- Pretrain CLI → Multi-task transformer: instantiates model (evidence: `src/dmc_pretrain.py`)
- Pretrain CLI → Test dataset: tests with (evidence: `src/dmc_pretrain.py`)
- Downstream CLI → DMC data modules: creates data module (evidence: `src/dmc_downstream.py`)
- Downstream CLI → Test dataset: tests with (evidence: `src/dmc_downstream.py`)
- Replay datasets → Azure data storage: downloaded from (evidence: `README.md`)
- DMC data modules → Replay buffer: uses buffer (evidence: `src/datamodules/dmc_datamodule.py`)
- DMC data modules → DMC environment factory: creates environments (evidence: `src/datamodules/dmc_datamodule.py`)
- DMC environment factory → Gym wrapper: registers wrapper (evidence: `src/envs/dmc2gym/dmc2gym/__init__.py`)
- Gym wrapper → DeepMind Control: loads tasks (evidence: `src/envs/dmc2gym/dmc2gym/wrappers.py`)
- Multi-task transformer → GPT backbone: uses backbone (evidence: `src/models/multitask_ct_module.py`)
- Control transformer → GPT backbone: uses backbone (evidence: `src/models/ct_module.py`)
- Multi-task transformer → Model utilities: gets action sizes (evidence: `src/models/multitask_ct_module.py`)
- Control transformer → Model utilities: gets action sizes (evidence: `src/models/ct_module.py`)
- DMC data modules → Model utilities: uses action helpers (evidence: `src/datamodules/dmc_datamodule.py`)
- Downstream CLI → Control transformer: selects configured model
- Multi-task transformer → DMC environment factory: evaluates policy
