# real-stanford/diffusion_policy architecture

> Architecture diagram of the GitHub repository real-stanford/diffusion_policy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/real-stanford/diffusion_policy
- Repository: https://github.com/real-stanford/diffusion_policy

## Overview

##### Project Type and Purpose
- The project is a robotics control framework using diffusion and transformer-based policies. It supports multiple tasks (e.g., block pushing, kitchen manipulation, and push-T) and accommodates both simulation and real-world scenarios.
- The README details both simulation and real robot operation, including special hardware requirements (notably a 3Dconnexion SpaceMouse for teleoperation).

##### Core Components from the File Structure
- Configurations: The “diffusion_policy/config” directory contains YAML files to configure both tasks (e.g., different variants of block pushing, kitchen tasks) and training pipelines (e.g., diffusion transformers, unet methods).
- Datasets and Replay Buffers: Within “diffusion_policy/dataset” you see dataset wrapper classes that load and preprocess demonstration data, including normalization routines.
- Environment Interfaces: Under “diffusion_policy/env” and “diffusion_policy/env_runner”, various environment wrappers exist for simulation (gym-based environments) and real-world experiments.
- Workspace & Training: In “diffusion_policy/workspace” the logic for launching training experiments is encapsulated. The “train.py” script brings these pieces together using Hydra for dynamic configuration.
- Shared Memory & Real-World Integration: The “diffusion_policy/shared_memory” and “diffusion_policy/real_world” directories incorporate modules for efficient data streaming (such as using SharedMemoryRingBuffer) from cameras and sensors. Within “diffusion_policy/real_world” you find “spacemouse.py” and “spacemouse_shared_memory.py”—key modules for mouse teleoperation control.
- External Data and Logging: Files like “multirun_metrics.py”, wandb integration in training scripts, and output directories show where logs, metrics, and checkpoints are managed.

##### Smart Mouse Control Module
- In the real-robot control chain, the SpaceMouse is used for teleoperation. The system interacts with the SpaceMouse via:
- The “spacemouse.py” module: This component likely reads raw inputs from the physical device.
- The “spacemouse_shared_memory.py” implementation: It passes the teleop commands via shared memory interfaces to ensure minimal latency.
- Smart Mouse Control module to the “RealEnv” component (or “Real Robot” wrapper) and further into the policy execution pipeline. For example, show bidirectional data flow: user inputs from the SpaceMouse trigger teleoperated motion commands which are incorporated into the policy’s real-time planning loop.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Simulation & Training"
        A["Configurations (Hydra)"]
        subgraph "Data Handling"
            B1["Dataset Manager"]
            B2["Replay Buffer"]
        end
        subgraph "Environment"
            C1["Gym Environment"]
            C2["Environment Runner"]
        end
        D["Policy Module"]
        subgraph "Training"
            E1["Training Workspace"]
            E2["Train Script"]
        end
        F["Logger (Wandb)"]
    end

    subgraph "Real-Robot Pipeline"
        G["Camera & Sensor Feed (Realsense)"]
        H["Shared Memory Integration"]
        I["Real Environment Interface"]
        subgraph "Smart Mouse Teleop"
            J1["SpaceMouse Input"]
            J2["Teleop Shared Memory"]
        end
    end

    %% Simulation & Training Flow
    A -->|"configures"| B1
    B1 -->|"loads"| B2
    B2 -->|"initiates"| C1
    C1 -->|"executes"| C2
    C2 -->|"sends_state"| D
    D -->|"updates"| E1
    E1 -->|"launches"| E2
    E1 -->|"logs"| F

    %% Real-Robot Pipeline Flow
    G -->|"feeds"| I
    H -->|"streams"| I
    J1 -->|"provides_input"| I
    J2 -->|"provides_input"| I
    I -->|"sends_state"| D
    D -->|"control_cmd"| I

    %% Click Events
    click A "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/config"
    click B1 "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/dataset"
    click B2 "https://github.com/real-stanford/diffusion_policy/blob/main/diffusion_policy/common/replay_buffer.py"
    click C1 "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/env"
    click C2 "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/env_runner"
    click D "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/policy"
    click E1 "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/workspace"
    click E2 "https://github.com/real-stanford/diffusion_policy/blob/main/train.py"
    click H "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/shared_memory"
    click I "https://github.com/real-stanford/diffusion_policy/tree/main/diffusion_policy/real_world"
    click J1 "https://github.com/real-stanford/diffusion_policy/blob/main/diffusion_policy/real_world/spacemouse.py"
    click J2 "https://github.com/real-stanford/diffusion_policy/blob/main/diffusion_policy/real_world/spacemouse_shared_memory.py"

    %% Styles
    classDef simulation fill:#add8e6,stroke:#000,stroke-width:2px;
    classDef real fill:#90ee90,stroke:#000,stroke-width:2px;
    classDef teleop fill:#ffa07a,stroke:#000,stroke-width:2px;

    class A,B1,B2,C1,C2,D,E1,E2,F simulation;
    class G,H,I real;
    class J1,J2 teleop;
```
