How hri-eu/flow_matching works
Project Type and Purpose
• This is a research-oriented Python framework for training and evaluating robot manipulation policies via flow matching on standard benchmarks (PushT, Franka Kitchen, Robomimic).
• It is not a web or full-stack app, but a CLI/config-driven library combining simulation environments, data loaders, model code, and training/evaluation scripts.
File Structure
Top-level:
– examples/ ▶ entry-point scripts that launch training or evaluation
– external/ ▶ vendored or sub-module code (diffusion_policy, robomimic, custom models)
– images/ ▶ static assets for documentation
– pkgInfo.py, requirements.txt, README.md, LICENSE.md
Under external/diffusion_policy:
– config/ ▶ YAML task and workspace definitions
– dataset/, env/, env_runner/, gym_util/ ▶ data loading, environment wrappers, runner abstractions
– model/, policy/, workspace/ ▶ core model architectures (diffusion nets, BET), policy wrappers, training workflows
Under external/robomimic:
– config/, envs/, algo/, models/, utils/, scripts/, tests/ ▶ baseline RL pipelines, datasets, utilities
Extract Insights from the README
• The pipeline graphic (images/overall.png) shows:
Config → Workspace → Runner → Environment & Dataset → Model → Policy → Evaluation
• Key dependencies: PyTorch, Gym, external diffusion_policy, robomimic libraries, MuJoCo/PyBullet environments, real-world cameras/controllers.
Main Components & Their Relationships
A. Configuration Layer
• YAML files under external/diffusion_policy/config/
• requirements.txt (specifies dependencies)
B. Orchestration Layer
• examples/*.py (scripts)
• pkgInfo.py (package metadata)
• Workspace classes (external/diffusion_policy/workspace/)
– parse configs, set up training pipelines
C. Runner Layer
• Env Runner classes (external/diffusion_policy/env_runner/)
– instantiate environments, dataset replay, stepping logic
D. Environment & Dataset Layer
• Simulated envs:
– PushT (external/diffusion_policy/env/pusht/)
– Franka Kitchen (external/diffusion_policy/env/kitchen/)
– Robomimic wrappers (external/diffusion_policy/env/robomimic/)
• Dataset loaders (external/diffusion_policy/dataset/)
• Gym utilities (async vectorization, video wrappers)
E. Model Layer
• Diffusion networks (external/diffusion_policy/model/diffusion/)
• BET networks (external/diffusion_policy/model/bet/)
• Vision encoders (external/diffusion_policy/model/vision/)
• Common utilities (lr schedulers, normalizers)
F. Policy Layer
• Policy wrappers (external/diffusion_policy/policy/)
– image vs low-dim policies, hybrid vs unet vs transformer
G. Real-World / Evaluation Layer
• Real-world interfaces (external/diffusion_policy/real_world/)
– multi-camera readers, RTDE controllers, video recorders
• Scripts for metrics (external/diffusion_policy/scripts/)
H. External Dependencies
• diffusion_policy (MIT-licensed vendored code)
• robomimic (separate pip/VCS package)
• TorchCFM, PyBullet, MuJoCo
Architectural Patterns & Principles
• Modular layering: config → orchestration → runner → env/dataset → model → policy → evaluation
• Plugin-style: new tasks can be added via YAML + env + policy class
• Separation of concerns:
– workspace focuses on setup
– runner focuses on loop logic
– env/dataset focuses on data sources
– model/policy focuses on computation