# hri-eu/flow_matching architecture

> Architecture diagram of the GitHub repository hri-eu/flow_matching, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hri-eu/flow_matching
- Repository: https://github.com/hri-eu/flow_matching

## Overview

##### Project Type and Purpose
- This is a research-oriented Python framework for training and evaluating robot manipulation policies via flow matching on standard benchmarks (PushT, Franka Kitchen, Robomimic).
- It is not a web or full-stack app, but a CLI/config-driven library combining simulation environments, data loaders, model code, and training/evaluation scripts.

##### File Structure
Top-level:
– examples/   ▶ entry-point scripts that launch training or evaluation
– external/   ▶ vendored or sub-module code (diffusion_policy, robomimic, custom models)
– images/    ▶ static assets for documentation
– pkgInfo.py, requirements.txt, README.md, LICENSE.md

Under external/diffusion_policy:
– config/   ▶ YAML task and workspace definitions
– dataset/, env/, env_runner/, gym_util/ ▶ data loading, environment wrappers, runner abstractions
– model/, policy/, workspace/ ▶ core model architectures (diffusion nets, BET), policy wrappers, training workflows

Under external/robomimic:
– config/, envs/, algo/, models/, utils/, scripts/, tests/ ▶ baseline RL pipelines, datasets, utilities

##### Extract Insights from the README
- The pipeline graphic (images/overall.png) shows:
Config → Workspace → Runner → Environment & Dataset → Model → Policy → Evaluation
- Key dependencies: PyTorch, Gym, external diffusion_policy, robomimic libraries, MuJoCo/PyBullet environments, real-world cameras/controllers.

##### Main Components & Their Relationships
A. Configuration Layer
- YAML files under external/diffusion_policy/config/
- requirements.txt (specifies dependencies)

B. Orchestration Layer
- examples/*.py (scripts)
- pkgInfo.py (package metadata)
- Workspace classes (external/diffusion_policy/workspace/)
– parse configs, set up training pipelines

C. Runner Layer
- Env Runner classes (external/diffusion_policy/env_runner/)
– instantiate environments, dataset replay, stepping logic

D. Environment & Dataset Layer
- Simulated envs:
– PushT (external/diffusion_policy/env/pusht/)
– Franka Kitchen (external/diffusion_policy/env/kitchen/)
– Robomimic wrappers (external/diffusion_policy/env/robomimic/)
- Dataset loaders (external/diffusion_policy/dataset/)
- Gym utilities (async vectorization, video wrappers)

E. Model Layer
- Diffusion networks (external/diffusion_policy/model/diffusion/)
- BET networks (external/diffusion_policy/model/bet/)
- Vision encoders (external/diffusion_policy/model/vision/)
- Common utilities (lr schedulers, normalizers)

F. Policy Layer
- Policy wrappers (external/diffusion_policy/policy/)
– image vs low-dim policies, hybrid vs unet vs transformer

G. Real-World / Evaluation Layer
- Real-world interfaces (external/diffusion_policy/real_world/)
– multi-camera readers, RTDE controllers, video recorders
- Scripts for metrics (external/diffusion_policy/scripts/)

H. External Dependencies
- diffusion_policy (MIT-licensed vendored code)
- robomimic (separate pip/VCS package)
- TorchCFM, PyBullet, MuJoCo

##### Architectural Patterns & Principles
- Modular layering: config → orchestration → runner → env/dataset → model → policy → evaluation
- Plugin-style: new tasks can be added via YAML + env + policy class
- Separation of concerns:
– workspace focuses on setup
– runner focuses on loop logic
– env/dataset focuses on data sources
– model/policy focuses on computation

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration Layer
    subgraph "Configuration Layer"
        CFG["Config YAML"]:::orange
        REQ["requirements.txt"]:::orange
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        EXAMPLES["examples/*.py"]:::blue
        PKG["pkgInfo.py"]:::blue
        WS["Workspace classes"]:::blue
    end

    %% Runner Layer
    subgraph "Runner Layer"
        RUNNER["Env Runner"]:::blue
    end

    %% Environment & Dataset Layer
    subgraph "Environment & Dataset Layer"
        subgraph "Environments"
            BP["BlockPush Env"]:::green
            KITCHEN["Kitchen Env"]:::green
            PUSHT["PushT Env"]:::green
            ROBOMIMIC_WRAPPER["Robomimic wrappers"]:::green
        end
        DATASET["Dataset loaders"]:::green
        GYMUTIL["Gym utilities"]:::green
    end

    %% Model Layer
    subgraph "Model Layer"
        DIFF["Diffusion networks"]:::blue
        BET["BET networks"]:::blue
        VISION["Vision encoders"]:::blue
        COMMON["Model common utilities"]:::blue
    end

    %% Policy Layer
    subgraph "Policy Layer"
        POLICY["Policy wrappers"]:::blue
    end

    %% Evaluation Layer
    subgraph "Evaluation Layer"
        REAL["Real-World interfaces"]:::blue
        SCRIPTS["Evaluation scripts"]:::orange
    end

    %% External Dependencies
    subgraph "External Dependencies"
        VENDOR["vendored diffusion_policy"]:::gray
        ROBO["robomimic library"]:::gray
        TORCH["TorchCFM, MuJoCo, PyBullet"]:::gray
    end

    %% Connections
    CFG -->|"config dict"| WS
    REQ -->|"dependencies"| WS
    EXAMPLES -->|"invoke"| WS
    PKG -->|"metadata"| WS

    WS -->|"setup runner"| RUNNER

    RUNNER -->|"instantiate"| BP
    RUNNER -->|"instantiate"| KITCHEN
    RUNNER -->|"instantiate"| PUSHT
    RUNNER -->|"instantiate"| ROBOMIMIC_WRAPPER
    RUNNER -->|"load batch"| DATASET
    RUNNER -->|"wrap env"| GYMUTIL

    RUNNER -->|"train call"| DIFF
    RUNNER -->|"train call"| BET
    RUNNER -->|"train call"| VISION
    RUNNER -->|"train call"| COMMON
    RUNNER -->|"train call"| POLICY

    DIFF -->|"features"| POLICY
    BET -->|"features"| POLICY
    VISION -->|"features"| POLICY
    COMMON -->|"utils"| POLICY

    POLICY -->|"policy action"| RUNNER

    RUNNER -->|"deploy actions"| REAL
    RUNNER -->|"metric logs"| SCRIPTS

    VENDOR --> WS
    VENDOR --> RUNNER
    VENDOR --> BP
    VENDOR --> DIFF
    VENDOR --> POLICY

    ROBO --> ROBOMIMIC_WRAPPER

    TORCH --> RUNNER
    TORCH --> DIFF
    TORCH --> BP

    %% Click Events
    click CFG "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/config/"
    click REQ "https://github.com/hri-eu/flow_matching/blob/master/requirements.txt"
    click EXAMPLES "https://github.com/hri-eu/flow_matching/blob/master/examples/flow_kitchen.py"
    click PKG "https://github.com/hri-eu/flow_matching/blob/master/pkgInfo.py"
    click WS "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/workspace/"
    click RUNNER "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/env_runner/"
    click BP "https://github.com/hri-eu/flow_matching/blob/master/external/diffusion_policy/env/block_pushing/block_pushing.py"
    click KITCHEN "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/env/kitchen/"
    click PUSHT "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/env/pusht/"
    click ROBOMIMIC_WRAPPER "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/env/robomimic/"
    click DATASET "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/dataset/"
    click GYMUTIL "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/gym_util/"
    click DIFF "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/model/diffusion/"
    click BET "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/model/bet/"
    click VISION "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/model/vision/"
    click COMMON "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/model/common/"
    click POLICY "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/policy/"
    click REAL "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/real_world/"
    click SCRIPTS "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/scripts/"
    click VENDOR "https://github.com/hri-eu/flow_matching/tree/master/external/diffusion_policy/"
    click ROBO "https://github.com/hri-eu/flow_matching/tree/master/external/robomimic/"

    %% Styles
    classDef blue fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef gray fill:#e0e0e0,stroke:#333,stroke-width:1px
    classDef green fill:#d4edda,stroke:#333,stroke-width:1px
    classDef orange fill:#ffe5b4,stroke:#333,stroke-width:1px
```
