# realrobotsquad/fastumi_data architecture

> Architecture diagram of the GitHub repository realrobotsquad/fastumi_data, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/realrobotsquad/fastumi_data
- Repository: https://github.com/realrobotsquad/fastumi_data

## Overview

##### Project Type & Purpose
- FastUMI is a robotics data acquisition and processing system that leverages ROS (Robot Operating System) to collect real-world manipulation trajectory data. The system is hardware-independent and accommodates different robot grippers while ensuring data consistency, making it directly applicable for robot learning and inverse kinematics (IK) training.
- Key functions include collecting raw sensor data (from RealSense T265, GoPro, etc.), transforming this data (e.g., applying offsets and coordinate transforms), and processing it for further use (such as conversions to Zarr format for Diffusion Policy-based training).

##### Major Components
A. Hardware and Sensor Integration:
- Sensors & Cameras:
- RealSense T265 and USB camera (GoPro), referenced by the launch files (e.g., usb_cam-test.launch) and detailed in the README.
- A URDF file (xarm6_robot.urdf in the assets folder) representing the robot’s model.
- ROS Environment:
- ROS core and associated nodes (e.g., realsense2_camera, usb_cam) that handle data streams.

B. Data Collection:
- Data Collection Scripts:
- data_collection.py (the core script starting data acquisition).
- Associated shell script (data_collection.sh) for launching the process.
- Data Organization:
- After collection, data is structured into directories (camera, csv, images inside a dataset folder) with temporal logs and raw trajectory recordings (temp_trajectory.csv, frame_timestamps.csv, etc.).

C. Data Processing:
- Processing Modules:
- data_processing_tcp_to_dp.py: Converts TCP data into Zarr format for training datasets.
- data_processing_to_joint.py & data_processing_to_tcp.py: Provide alternative processing pipelines to compute inverse kinematics and transform trajectories.
- Datatool Package:
- Contains modules (data_image.py, data_trajectory.py) that likely offer reusable utilities for handling image and trajectory data.

D. Configuration & Documentation:
- Config Files:
- config/config.json and config/readme.md define transformation offsets (e.g., TCP transformation, gripper width scaling) and sensor calibration parameters.
- Documentation:
- The extensive README and docs provide workflow steps, environment setup, and even physical prototypes of the system.

E. Data Storage and Output:
- HDF5 Files:
- Episode data stored in HDF5 files (episode_*.hdf5) include hierarchical data (observations with images and qpos, actions, and attributes).
- CSV Files:
- Logs that capture state, trajectories, and timestamps.
- Other storage tools like Zarr provide more flexible storage for processed datasets.

##### Relationships & Interactions
- The hardware sensors provide raw data to the ROS nodes (RealSense T265 and USB camera nodes), which then stream into the data collection scripts.
- The data_collection.py script organizes and logs data (images, trajectories, timestamps) into a structured file system.
- Once the raw data is collected, dedicated processing scripts transform and compute additional data (e.g., inverse kinematics, coordinate transformations, gripper detection) leveraging configuration parameters from config files.
- The datatool modules act as utility layers facilitating consistent data handling across the board.
- Finally, processed data packages (e.g., HDF5 and Zarr datasets) are output for subsequent use in robot policy training.

##### Architectural Patterns & Design Principles
- Modular Design: Separate scripts/modules handle data collection, processing, and utilities, allowing for clear separation of concerns.
- Decoupled Data Processing Pipeline: The raw data capture is separated from the processing stage, allowing flexibility for integration with different robot models and data formats.
- Hardware Abstraction: The use of ROS and standard URDF files abstract the physical hardware specifics, making the system broadly compatible.
- Data Transformation & Calibration: Configuration files ensure that sensor data is appropriately transformed (using specific offsets, gripper detection calibration) before further processing.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Hardware Layer
    subgraph "Hardware Layer"
        RS("RealSense T265 Sensor"):::sensor
        USB("USB/GoPro Camera"):::sensor
        Robot("Robot Model"):::sensor
    end

    %% ROS Environment
    subgraph "ROS Environment"
        ROS("ROS Core"):::middleware
        RSNode("RealSense Node"):::middleware
        USBCamNode("USB Cam Node"):::middleware
    end

    %% Data Collection Module
    subgraph "Data Collection"
        DCS("Data Collection Script"):::collection
        DCShell("Data Collection Shell Script"):::collection
        DS("Dataset Organization"):::storage
    end

    %% Data Processing Module
    subgraph "Data Processing"
        TCPZ("TCP->Zarr Processing"):::processing
        JointP("To Joint Processing"):::processing
        TCPTrans("To TCP Processing"):::processing
        Datatool("Datatool Package"):::processing
    end

    %% Configuration & Documentation
    subgraph "Configuration & Documentation"
        Config("Configuration Files"):::config
        Docs("Documentation"):::config
    end

    %% Data Storage & Output
    subgraph "Data Storage & Output"
        HDF5("HDF5 Files"):::storage
        CSV("CSV Logs"):::storage
        Zarr("Zarr Datasets"):::storage
    end

    %% Connections from Hardware to ROS
    RS -->|"streams data"| RSNode
    USB -->|"streams data"| USBCamNode
    Robot -->|"provides model"| ROS

    %% ROS integration
    RSNode -->|"feeds sensor data"| ROS
    USBCamNode -->|"feeds sensor data"| ROS

    %% ROS to Data Collection
    ROS -->|"data pipeline"| DCS
    ROS -->|"data pipeline"| DCShell

    %% Data Collection to Dataset Organization
    DCS -->|"organizes data"| DS
    DCShell -->|"initiates collection"| DS

    %% Data Collection to Data Processing
    DS -->|"raw data"| TCPZ
    DS -->|"raw data"| JointP
    DS -->|"raw data"| TCPTrans

    %% Configuration influence
    Config -->|"parameters"| DCS
    Config -->|"parameters"| TCPZ
    Config -->|"parameters"| JointP
    Config -->|"parameters"| TCPTrans

    %% Data Processing uses utilities
    Datatool -->|"assists processing"| TCPZ
    Datatool -->|"assists processing"| JointP
    Datatool -->|"assists processing"| TCPTrans

    %% Data Processing to Data Storage
    TCPZ -->|"processed output"| Zarr
    JointP -->|"processed output"| HDF5
    TCPTrans -->|"processed output"| CSV

    %% Documentation supports all layers
    Docs ---|"guides configuration and setup"| Config

    %% Click Events for Component Mapping
    click Robot "https://github.com/realrobotsquad/fastumi_data/blob/main/assets/xarm6_robot.urdf"
    click USB "https://github.com/realrobotsquad/fastumi_data/blob/main/usb_cam-test.launch"
    click DCS "https://github.com/realrobotsquad/fastumi_data/blob/main/data_collection.py"
    click DCShell "https://github.com/realrobotsquad/fastumi_data/blob/main/data_collection.sh"
    click TCPZ "https://github.com/realrobotsquad/fastumi_data/blob/main/data_processing_tcp_to_dp.py"
    click JointP "https://github.com/realrobotsquad/fastumi_data/blob/main/data_processing_to_joint.py"
    click TCPTrans "https://github.com/realrobotsquad/fastumi_data/blob/main/data_processing_to_tcp.py"
    click Datatool "https://github.com/realrobotsquad/fastumi_data/tree/main/datatool/"
    click Config "https://github.com/realrobotsquad/fastumi_data/tree/main/config/"
    click Docs "https://github.com/realrobotsquad/fastumi_data/blob/main/README.md"

    %% Styles
    classDef sensor fill:#ffeda0,stroke:#e89c3d,stroke-width:2px;
    classDef middleware fill:#c7e9c0,stroke:#66c2a5,stroke-width:2px;
    classDef collection fill:#9ecae1,stroke:#3182bd,stroke-width:2px;
    classDef processing fill:#fdd0a2,stroke:#fdae6b,stroke-width:2px;
    classDef config fill:#dadaeb,stroke:#9e9ac8,stroke-width:2px;
    classDef storage fill:#fed9a6,stroke:#fd8d3c,stroke-width:2px;
```
