# dexmal/opendm architecture

> Architecture diagram of the GitHub repository dexmal/opendm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dexmal/opendm
- Repository: https://github.com/dexmal/opendm

## Overview

OpenDM is a vision-language-action model toolkit for training and deploying DM0.5 across simulated and physical robot embodiments. The main workflow is to configure a task recipe, load and transform robot demonstrations, fine-tune the model, then serve policy inference for robot observations. The graph also includes dataset registration, evaluation-oriented recipes, and the optimized TensorRT inference path. Relationships among sampled modules follow the source index or excerpts; broader evaluation and deployment capabilities are documented in the README, while unsampled implementation details are not inferred.

Main entry points: [`opendm/exp/dm05_exp.py`](https://github.com/dexmal/opendm/blob/main/opendm/exp/dm05_exp.py) (DM05 Experiment), [`opendm/data/transforms.py`](https://github.com/dexmal/opendm/blob/main/opendm/data/transforms.py) (Data Transforms), [`opendm/infer/dm05_infer.py`](https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_infer.py) (Inference Service), [`playground/dm05_robotwin2.py`](https://github.com/dexmal/opendm/blob/main/playground/dm05_robotwin2.py) (Task Recipe)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_recipes["Task Recipes"]
  node_task_recipe["Task Recipe<br/>[dm05_robotwin2.py]"]
  node_evaluation_recipes["Evaluation Recipes"]
end

subgraph group_data["Robot Data"]
  node_dataset_registry["Dataset Registry<br/>[register.py]"]
  node_robot_datasets["Robot Datasets"]
  node_jsonl_dataset["JSONL Dataset<br/>[dataset.py]"]
  node_data_transforms["Data Transforms<br/>[transforms.py]"]
  node_normalization["State Normalization<br/>[normalize.py]"]
  node_image_augmentation["Image Augmentation<br/>[augmentations.py]"]
  node_batch_collator["Batch Collator<br/>[collator.py]"]
  node_robot_constants["Robot Specifications<br/>[robot.py]"]
end

subgraph group_training["Model Training"]
  node_experiment["DM05 Experiment<br/>[dm05_exp.py]"]
  node_dm05_model["DM05 Model<br/>[dm05_arch.py]"]
  node_lora["LoRA Fine-tuning<br/>[dm05_lora.py]"]
  node_optimizer["Optimizer<br/>[muon_adamw.py]"]
  node_trainer["Trainer<br/>[trainer.py]"]
end

subgraph group_runtime["Policy Runtime"]
  node_inference_service["Inference Service<br/>[dm05_infer.py]"]
  node_fast_inference["Fast Decode<br/>[dm05_infer_arch.py]"]
  node_trt_vision["TensorRT Vision"]
  node_fp8_prefix["FP8 Prefix Backend"]
end

node_researcher(("Researcher"))
node_robot_client(("Robot Client"))

node_researcher -->|"configures"| node_task_recipe
node_researcher -->|"evaluates"| node_evaluation_recipes
node_task_recipe -->|"configures"| node_experiment
node_evaluation_recipes -->|"configures"| node_experiment
node_dataset_registry -->|"imports"| node_robot_datasets
node_task_recipe -->|"registers"| node_dataset_registry
node_experiment -->|"uses"| node_dataset_registry
node_experiment -->|"uses"| node_jsonl_dataset
node_experiment -->|"builds inputs with"| node_data_transforms
node_experiment -->|"batches with"| node_batch_collator
node_data_transforms -->|"uses"| node_normalization
node_data_transforms -->|"uses"| node_image_augmentation
node_data_transforms -->|"uses"| node_robot_constants
node_experiment -->|"configures"| node_dm05_model
node_experiment -->|"uses"| node_lora
node_experiment -->|"configures"| node_optimizer
node_experiment -->|"runs"| node_trainer
node_robot_client -->|"sends observations"| node_inference_service
node_inference_service -->|"dispatches decode"| node_fast_inference
node_inference_service -->|"runs vision engine"| node_trt_vision
node_inference_service -.->|"optionally uses"| node_fp8_prefix
node_fast_inference -->|"uses model"| node_dm05_model

click node_task_recipe "https://github.com/dexmal/opendm/blob/main/playground/dm05_robotwin2.py"
click node_evaluation_recipes "https://github.com/dexmal/opendm/tree/main/playground"
click node_dataset_registry "https://github.com/dexmal/opendm/blob/main/opendm/dataset/register.py"
click node_robot_datasets "https://github.com/dexmal/opendm/tree/main/opendm/dataset"
click node_jsonl_dataset "https://github.com/dexmal/opendm/blob/main/opendm/data/dataset.py"
click node_data_transforms "https://github.com/dexmal/opendm/blob/main/opendm/data/transforms.py"
click node_normalization "https://github.com/dexmal/opendm/blob/main/opendm/data/normalize.py"
click node_image_augmentation "https://github.com/dexmal/opendm/blob/main/opendm/data/augmentations.py"
click node_batch_collator "https://github.com/dexmal/opendm/blob/main/opendm/data/collator.py"
click node_experiment "https://github.com/dexmal/opendm/blob/main/opendm/exp/dm05_exp.py"
click node_dm05_model "https://github.com/dexmal/opendm/blob/main/opendm/model/dm05/dm05_arch.py"
click node_lora "https://github.com/dexmal/opendm/blob/main/opendm/model/dm05/dm05_lora.py"
click node_optimizer "https://github.com/dexmal/opendm/blob/main/opendm/optimizer/muon_adamw.py"
click node_trainer "https://github.com/dexmal/opendm/blob/main/opendm/trainer/trainer.py"
click node_inference_service "https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_infer.py"
click node_fast_inference "https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_infer_arch.py"
click node_trt_vision "https://github.com/dexmal/opendm/blob/main/opendm/infer/build_vision_trt.py"
click node_fp8_prefix "https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_prefix_mlp_trt.py"
click node_robot_constants "https://github.com/dexmal/opendm/blob/main/opendm/constants/robot.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_task_recipe,node_evaluation_recipes,node_robot_client toneBlue
class node_dataset_registry,node_robot_datasets,node_jsonl_dataset,node_data_transforms,node_normalization,node_image_augmentation,node_batch_collator,node_robot_constants toneAmber
class node_experiment,node_dm05_model,node_lora,node_optimizer,node_trainer toneMint
class node_inference_service,node_fast_inference,node_trt_vision,node_fp8_prefix toneRose
class node_researcher toneIndigo
```

## Components

### Task Recipes

- [Task Recipe](https://github.com/dexmal/opendm/blob/main/playground/dm05_robotwin2.py): `playground/dm05_robotwin2.py`
- [Evaluation Recipes](https://github.com/dexmal/opendm/tree/main/playground): `playground`

### Robot Data

- [Dataset Registry](https://github.com/dexmal/opendm/blob/main/opendm/dataset/register.py): `opendm/dataset/register.py`
- [Robot Datasets](https://github.com/dexmal/opendm/tree/main/opendm/dataset): `opendm/dataset`
- [JSONL Dataset](https://github.com/dexmal/opendm/blob/main/opendm/data/dataset.py): `opendm/data/dataset.py`
- [Data Transforms](https://github.com/dexmal/opendm/blob/main/opendm/data/transforms.py): `opendm/data/transforms.py`
- [State Normalization](https://github.com/dexmal/opendm/blob/main/opendm/data/normalize.py): `opendm/data/normalize.py`
- [Image Augmentation](https://github.com/dexmal/opendm/blob/main/opendm/data/augmentations.py): `opendm/data/augmentations.py`
- [Batch Collator](https://github.com/dexmal/opendm/blob/main/opendm/data/collator.py): `opendm/data/collator.py`
- [Robot Specifications](https://github.com/dexmal/opendm/blob/main/opendm/constants/robot.py): `opendm/constants/robot.py`

### Model Training

- [DM05 Experiment](https://github.com/dexmal/opendm/blob/main/opendm/exp/dm05_exp.py): `opendm/exp/dm05_exp.py`
- [DM05 Model](https://github.com/dexmal/opendm/blob/main/opendm/model/dm05/dm05_arch.py): `opendm/model/dm05/dm05_arch.py`
- [LoRA Fine-tuning](https://github.com/dexmal/opendm/blob/main/opendm/model/dm05/dm05_lora.py): `opendm/model/dm05/dm05_lora.py`
- [Optimizer](https://github.com/dexmal/opendm/blob/main/opendm/optimizer/muon_adamw.py): `opendm/optimizer/muon_adamw.py`
- [Trainer](https://github.com/dexmal/opendm/blob/main/opendm/trainer/trainer.py): `opendm/trainer/trainer.py`

### Policy Runtime

- [Inference Service](https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_infer.py): `opendm/infer/dm05_infer.py`
- [Fast Decode](https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_infer_arch.py): `opendm/infer/dm05_infer_arch.py`
- [TensorRT Vision](https://github.com/dexmal/opendm/blob/main/opendm/infer/build_vision_trt.py): `opendm/infer/build_vision_trt.py`
- [FP8 Prefix Backend](https://github.com/dexmal/opendm/blob/main/opendm/infer/dm05_prefix_mlp_trt.py): `opendm/infer/dm05_prefix_mlp_trt.py`

### Other components

- **Researcher**
- **Robot Client**

## Connections

- Researcher → Task Recipe: configures
- Researcher → Evaluation Recipes: evaluates
- Task Recipe → DM05 Experiment: configures (evidence: `playground/dm05_robotwin2.py`)
- Evaluation Recipes → DM05 Experiment: configures (evidence: `playground/dm05_vla_arena.py`)
- Dataset Registry → Robot Datasets: imports (evidence: `opendm/dataset/register.py`)
- Task Recipe → Dataset Registry: registers (evidence: `playground/dm05_robotwin2.py`)
- DM05 Experiment → Dataset Registry: uses (evidence: `opendm/exp/dm05_exp.py`)
- DM05 Experiment → JSONL Dataset: uses (evidence: `opendm/exp/dm05_exp.py`)
- DM05 Experiment → Data Transforms: builds inputs with (evidence: `opendm/exp/dm05_exp.py`)
- DM05 Experiment → Batch Collator: batches with (evidence: `opendm/exp/dm05_exp.py`)
- Data Transforms → State Normalization: uses (evidence: `opendm/data/transforms.py`)
- Data Transforms → Image Augmentation: uses (evidence: `opendm/data/transforms.py`)
- Data Transforms → Robot Specifications: uses (evidence: `opendm/data/transforms.py`)
- DM05 Experiment → DM05 Model: configures (evidence: `opendm/exp/dm05_exp.py`)
- DM05 Experiment → LoRA Fine-tuning: uses (evidence: `opendm/exp/dm05_exp.py`)
- DM05 Experiment → Optimizer: configures (evidence: `opendm/exp/dm05_exp.py`)
- DM05 Experiment → Trainer: runs (evidence: `opendm/exp/dm05_exp.py`)
- Robot Client → Inference Service: sends observations
- Inference Service → Fast Decode: dispatches decode (evidence: `opendm/infer/dm05_infer.py`)
- Inference Service → TensorRT Vision: runs vision engine (evidence: `opendm/infer/dm05_infer.py`)
- Inference Service → FP8 Prefix Backend: optionally uses (evidence: `opendm/infer/dm05_infer.py`)
- Fast Decode → DM05 Model: uses model (evidence: `opendm/infer/dm05_infer_arch.py`)
