How realrobotsquad/fastumi_data works
Project Type & Purpose
• FastUMI is a robotics data acquisition and processing system that leverages ROS (Robot Operating System) to collect real-world manipulation trajectory data. The system is hardware-independent and accommodates different robot grippers while ensuring data consistency, making it directly applicable for robot learning and inverse kinematics (IK) training.
• Key functions include collecting raw sensor data (from RealSense T265, GoPro, etc.), transforming this data (e.g., applying offsets and coordinate transforms), and processing it for further use (such as conversions to Zarr format for Diffusion Policy-based training).
Major Components
A. Hardware and Sensor Integration:
• Sensors & Cameras:
• RealSense T265 and USB camera (GoPro), referenced by the launch files (e.g., usb_cam-test.launch) and detailed in the README.
• A URDF file (xarm6_robot.urdf in the assets folder) representing the robot’s model.
• ROS Environment:
• ROS core and associated nodes (e.g., realsense2_camera, usb_cam) that handle data streams.
B. Data Collection:
• Data Collection Scripts:
• data_collection.py (the core script starting data acquisition).
• Associated shell script (data_collection.sh) for launching the process.
• Data Organization:
• After collection, data is structured into directories (camera, csv, images inside a dataset folder) with temporal logs and raw trajectory recordings (temp_trajectory.csv, frame_timestamps.csv, etc.).
C. Data Processing:
• Processing Modules:
• data_processing_tcp_to_dp.py: Converts TCP data into Zarr format for training datasets.
• data_processing_to_joint.py & data_processing_to_tcp.py: Provide alternative processing pipelines to compute inverse kinematics and transform trajectories.
• Datatool Package:
• Contains modules (data_image.py, data_trajectory.py) that likely offer reusable utilities for handling image and trajectory data.
D. Configuration & Documentation:
• Config Files:
• config/config.json and config/readme.md define transformation offsets (e.g., TCP transformation, gripper width scaling) and sensor calibration parameters.
• Documentation:
• The extensive README and docs provide workflow steps, environment setup, and even physical prototypes of the system.
E. Data Storage and Output:
• HDF5 Files:
• Episode data stored in HDF5 files (episode_*.hdf5) include hierarchical data (observations with images and qpos, actions, and attributes).
• CSV Files:
• Logs that capture state, trajectories, and timestamps.
• Other storage tools like Zarr provide more flexible storage for processed datasets.
Relationships & Interactions
• The hardware sensors provide raw data to the ROS nodes (RealSense T265 and USB camera nodes), which then stream into the data collection scripts.
• The data_collection.py script organizes and logs data (images, trajectories, timestamps) into a structured file system.
• Once the raw data is collected, dedicated processing scripts transform and compute additional data (e.g., inverse kinematics, coordinate transformations, gripper detection) leveraging configuration parameters from config files.
• The datatool modules act as utility layers facilitating consistent data handling across the board.
• Finally, processed data packages (e.g., HDF5 and Zarr datasets) are output for subsequent use in robot policy training.
Architectural Patterns & Design Principles
• Modular Design: Separate scripts/modules handle data collection, processing, and utilities, allowing for clear separation of concerns.
• Decoupled Data Processing Pipeline: The raw data capture is separated from the processing stage, allowing flexibility for integration with different robot models and data formats.
• Hardware Abstraction: The use of ROS and standard URDF files abstract the physical hardware specifics, making the system broadly compatible.
• Data Transformation & Calibration: Configuration files ensure that sensor data is appropriately transformed (using specific offsets, gripper detection calibration) before further processing.