# mondo-robotics/dit4dit architecture

> Architecture diagram of the GitHub repository mondo-robotics/dit4dit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mondo-robotics/dit4dit
- Repository: https://github.com/mondo-robotics/dit4dit

## Overview

DiT4DiT is a vision-action model for tabletop and humanoid manipulation. Its principal workflow loads LeRobot demonstrations, transforms video/state/action data, trains a video-and-action model, then serves predictions for robot control. The repository also includes real-G1 sensing and control, model-serving transports, and a bundled Unitree SDK. The graph connects only relationships supported by the README or SOURCE INDEX references; some subsystem links use the README where implementation excerpts were unavailable.

Main entry points: [`DiT4DiT/dataloader/gr00t_lerobot/datasets.py`](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/datasets.py) (LeRobot datasets), [`DiT4DiT/model/framework/DiT4DiT.py`](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/DiT4DiT.py) (DiT4DiT framework), [`decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py`](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py) (G1 inference loop), [`DiT4DiT/training/train.py`](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/train.py) (Training entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data ingestion"]
  node_dataset["LeRobot datasets<br/>[datasets.py]"]
  node_schema["Data schema<br/>[schema.py]"]
  node_embodiment["Embodiment tags<br/>[embodiment_tags.py]"]
  node_transform["Transform base<br/>[base.py]"]
  node_stateaction["State/action transforms<br/>[state_action.py]"]
  node_videotransform["Video transforms<br/>[video.py]"]
  node_video["Video loading<br/>[video.py]"]
end

subgraph group_model["Model and training"]
  node_learner["Training entry<br/>[train.py]"]
  node_framework["DiT4DiT framework<br/>[DiT4DiT.py]"]
  node_baseframework["Framework base<br/>[base_framework.py]"]
  node_sharetools["Shared model tools<br/>[share_tools.py]"]
  node_cosmos["Cosmos backbone<br/>[Cosmos25.py]"]
  node_actiondit["Action DiT<br/>[ActionDiT.py]"]
  node_crossattention["Flow matching head"]
  node_modeltools["Model utilities<br/>[tools.py]"]
  node_trainerutils["Trainer utilities<br/>[trainer_tools.py]"]
  node_configtracker["Config tracking<br/>[config_tracker.py]"]
end

subgraph group_serving["Inference serving"]
  node_zmqserver["ZMQ policy server"]
end

subgraph group_robot["Robot control"]
  node_service["Inference service<br/>[service.py]"]
  node_inference["G1 inference loop"]
  node_camera["Composed camera<br/>[composed_camera.py]"]
end

subgraph group_sdk["Unitree SDK"]
  node_rpcclient["RPC client<br/>[client.py]"]
  node_rpcinternal["RPC internals<br/>[internal.py]"]
  node_sportclient["Sport client<br/>[sport_client.py]"]
end

node_robot(("Robot operator"))
node_g1["Unitree G1"]

node_learner -->|"loads batches"| node_dataset
node_learner -->|"trains model"| node_framework
node_learner -->|"uses utilities"| node_trainerutils
node_learner -->|"tracks config"| node_configtracker
node_dataset -->|"loads video"| node_video
node_dataset -->|"uses schema"| node_schema
node_dataset -->|"uses tags"| node_embodiment
node_dataset -->|"uses transforms"| node_transform
node_transform -->|"uses schema"| node_schema
node_stateaction -->|"extends base"| node_transform
node_stateaction -->|"uses schema"| node_schema
node_framework -->|"extends framework"| node_baseframework
node_framework -->|"uses backbone"| node_cosmos
node_framework -->|"uses action model"| node_actiondit
node_framework -->|"uses utilities"| node_trainerutils
node_baseframework -->|"uses shared tools"| node_sharetools
node_baseframework -->|"uses utilities"| node_modeltools
node_actiondit -->|"uses head"| node_crossattention
node_actiondit -->|"uses utilities"| node_modeltools
node_robot -.->|"trains policy"| node_learner
node_zmqserver -->|"loads framework"| node_baseframework
node_zmqserver -->|"uses shared tools"| node_sharetools
node_robot -.->|"starts control"| node_inference
node_inference -.->|"requests inference"| node_service
node_inference -->|"reads images"| node_camera
node_inference -.->|"requests actions"| node_zmqserver
node_inference -.->|"publishes actions"| node_g1
node_sportclient -->|"calls RPC"| node_rpcclient
node_rpcclient -->|"uses internals"| node_rpcinternal
node_sportclient -.->|"controls robot"| node_g1

click node_learner "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/train.py"
click node_dataset "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/datasets.py"
click node_schema "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/schema.py"
click node_embodiment "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/embodiment_tags.py"
click node_transform "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/transform/base.py"
click node_stateaction "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/transform/state_action.py"
click node_videotransform "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/transform/video.py"
click node_video "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/video.py"
click node_framework "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/DiT4DiT.py"
click node_baseframework "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/base_framework.py"
click node_sharetools "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/share_tools.py"
click node_cosmos "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/modules/vlm/Cosmos25.py"
click node_actiondit "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/modules/action_model/ActionDiT.py"
click node_crossattention "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/modules/action_model/flow_matching_head/cross_attention_dit.py"
click node_modeltools "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/tools.py"
click node_trainerutils "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/trainer_utils/trainer_tools.py"
click node_configtracker "https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/trainer_utils/config_tracker.py"
click node_zmqserver "https://github.com/mondo-robotics/dit4dit/blob/main/deployment/model_server/server_policy_zmq.py"
click node_service "https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/utils/service.py"
click node_inference "https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py"
click node_camera "https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/sensor/composed_camera.py"
click node_rpcclient "https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/client.py"
click node_rpcinternal "https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/internal.py"
click node_sportclient "https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/go2/sport/sport_client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset,node_schema,node_embodiment,node_transform,node_stateaction,node_videotransform,node_video toneBlue
class node_learner,node_framework,node_baseframework,node_sharetools,node_cosmos,node_actiondit,node_crossattention,node_modeltools,node_trainerutils,node_configtracker toneAmber
class node_zmqserver toneMint
class node_service,node_inference,node_camera toneRose
class node_rpcclient,node_rpcinternal,node_sportclient,node_robot,node_g1 toneIndigo
```

## Components

### Data ingestion

- [LeRobot datasets](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/datasets.py): `DiT4DiT/dataloader/gr00t_lerobot/datasets.py`
- [Data schema](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/schema.py): `DiT4DiT/dataloader/gr00t_lerobot/schema.py`
- [Embodiment tags](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/embodiment_tags.py): `DiT4DiT/dataloader/gr00t_lerobot/embodiment_tags.py`
- [Transform base](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/transform/base.py): `DiT4DiT/dataloader/gr00t_lerobot/transform/base.py`
- [State/action transforms](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/transform/state_action.py): `DiT4DiT/dataloader/gr00t_lerobot/transform/state_action.py`
- [Video transforms](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/transform/video.py): `DiT4DiT/dataloader/gr00t_lerobot/transform/video.py`
- [Video loading](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/dataloader/gr00t_lerobot/video.py): `DiT4DiT/dataloader/gr00t_lerobot/video.py`

### Model and training

- [Training entry](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/train.py): `DiT4DiT/training/train.py`
- [DiT4DiT framework](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/DiT4DiT.py): `DiT4DiT/model/framework/DiT4DiT.py`
- [Framework base](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/base_framework.py): `DiT4DiT/model/framework/base_framework.py`
- [Shared model tools](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/framework/share_tools.py): `DiT4DiT/model/framework/share_tools.py`
- [Cosmos backbone](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/modules/vlm/Cosmos25.py): `DiT4DiT/model/modules/vlm/Cosmos25.py`
- [Action DiT](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/modules/action_model/ActionDiT.py): `DiT4DiT/model/modules/action_model/ActionDiT.py`
- [Flow matching head](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/modules/action_model/flow_matching_head/cross_attention_dit.py): `DiT4DiT/model/modules/action_model/flow_matching_head/cross_attention_dit.py`
- [Model utilities](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/model/tools.py): `DiT4DiT/model/tools.py`
- [Trainer utilities](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/trainer_utils/trainer_tools.py): `DiT4DiT/training/trainer_utils/trainer_tools.py`
- [Config tracking](https://github.com/mondo-robotics/dit4dit/blob/main/DiT4DiT/training/trainer_utils/config_tracker.py): `DiT4DiT/training/trainer_utils/config_tracker.py`

### Inference serving

- [ZMQ policy server](https://github.com/mondo-robotics/dit4dit/blob/main/deployment/model_server/server_policy_zmq.py): `deployment/model_server/server_policy_zmq.py`

### Robot control

- [Inference service](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/utils/service.py): `decoupled_wbc/gr00t_wbc/control/utils/service.py`
- [G1 inference loop](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py): `decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py`
- [Composed camera](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/gr00t_wbc/control/sensor/composed_camera.py): `decoupled_wbc/gr00t_wbc/control/sensor/composed_camera.py`

### Unitree SDK

- [RPC client](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/client.py): `decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/client.py`
- [RPC internals](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/internal.py): `decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/internal.py`
- [Sport client](https://github.com/mondo-robotics/dit4dit/blob/main/decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/go2/sport/sport_client.py): `decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/go2/sport/sport_client.py`

### Other components

- **Robot operator**
- **Unitree G1**

## Connections

- Training entry → LeRobot datasets: loads batches (evidence: `DiT4DiT/training/train.py`)
- Training entry → DiT4DiT framework: trains model (evidence: `DiT4DiT/training/train.py`)
- Training entry → Trainer utilities: uses utilities (evidence: `DiT4DiT/training/train.py`)
- Training entry → Config tracking: tracks config (evidence: `DiT4DiT/training/train.py`)
- LeRobot datasets → Video loading: loads video (evidence: `DiT4DiT/dataloader/gr00t_lerobot/datasets.py`)
- LeRobot datasets → Data schema: uses schema (evidence: `DiT4DiT/dataloader/gr00t_lerobot/datasets.py`)
- LeRobot datasets → Embodiment tags: uses tags (evidence: `DiT4DiT/dataloader/gr00t_lerobot/datasets.py`)
- LeRobot datasets → Transform base: uses transforms (evidence: `DiT4DiT/dataloader/gr00t_lerobot/datasets.py`)
- Transform base → Data schema: uses schema (evidence: `DiT4DiT/dataloader/gr00t_lerobot/transform/base.py`)
- State/action transforms → Transform base: extends base (evidence: `DiT4DiT/dataloader/gr00t_lerobot/transform/state_action.py`)
- State/action transforms → Data schema: uses schema (evidence: `DiT4DiT/dataloader/gr00t_lerobot/transform/state_action.py`)
- DiT4DiT framework → Framework base: extends framework (evidence: `DiT4DiT/model/framework/DiT4DiT.py`)
- DiT4DiT framework → Cosmos backbone: uses backbone (evidence: `DiT4DiT/model/framework/DiT4DiT.py`)
- DiT4DiT framework → Action DiT: uses action model (evidence: `DiT4DiT/model/framework/DiT4DiT.py`)
- DiT4DiT framework → Trainer utilities: uses utilities (evidence: `DiT4DiT/model/framework/DiT4DiT.py`)
- Framework base → Shared model tools: uses shared tools (evidence: `DiT4DiT/model/framework/base_framework.py`)
- Framework base → Model utilities: uses utilities (evidence: `DiT4DiT/model/framework/base_framework.py`)
- Action DiT → Flow matching head: uses head (evidence: `DiT4DiT/model/modules/action_model/ActionDiT.py`)
- Action DiT → Model utilities: uses utilities (evidence: `DiT4DiT/model/framework/DiT4DiT.py`)
- Robot operator → Training entry: trains policy (evidence: `README.md`)
- ZMQ policy server → Framework base: loads framework (evidence: `deployment/model_server/server_policy_zmq.py`)
- ZMQ policy server → Shared model tools: uses shared tools (evidence: `deployment/model_server/server_policy_zmq.py`)
- Robot operator → G1 inference loop: starts control
- G1 inference loop → Inference service: requests inference (evidence: `decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py`)
- G1 inference loop → Composed camera: reads images (evidence: `decoupled_wbc/gr00t_wbc/control/main/teleop/run_dit4dit_inference_policy_loop.py`)
- G1 inference loop → ZMQ policy server: requests actions (evidence: `README.md`)
- G1 inference loop → Unitree G1: publishes actions (evidence: `README.md`)
- Sport client → RPC client: calls RPC (evidence: `decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/go2/sport/sport_client.py`)
- RPC client → RPC internals: uses internals (evidence: `decoupled_wbc/external_dependencies/unitree_sdk2_python/unitree_sdk2py/rpc/client.py`)
- Sport client → Unitree G1: controls robot
