How droid-dataset/droid works
Project Type and Purpose
• The project is a platform for setting up a teleoperated robot system to collect demonstration data – “in-the-wild” dataset collection for robot manipulation.
• Its purpose is twofold: (a) establishing the hardware–software integration for the robot (including camera calibration, sensor input, and robot control such as with the Franka arm), and (b) running workflows for data collection, evaluation, postprocessing, and training of policies.
File Structure Analysis
• Notice the presence of multiple Docker-related directories (.docker) with configurations for different environments (“laptop” and “nuc”). This shows an emphasis on containerized deployment.
• The “droid” directory encapsulates the core functionality. It includes many submodules – for calibration, camera utilities (with support for live cameras and recordings), controllers (including an Oculus controller), data loading and processing, evaluation routines, robot and inverse kinematics (robot_ik), training, postprocessing and user interface components.
• There is also a “scripts” directory that provides automation for tasks (e.g. data conversion, evaluation, training, testing), which supports the operational workflows.
• Configuration files (in the “config” directory) ensure hardware-specific parameters are clearly separated (e.g., using “fr3” and “panda” setups).
Key Components and Their Roles
a. Hardware & External Devices:
– Robot hardware (e.g., Franka robotic arms tied to “droid/franka” and “droid/robot_ik”).
– Cameras (via modules in “droid/camera_utils”).
– Controllers (including “droid/controllers/oculus_controller.py” for teleoperation).
b. Data Acquisition & User Interface:
– Data collection is managed in “droid/user_interface” (with GUIs and data collector scripts) and supplementary scripts.
– Camera readers (both live: “zed_camera.py”, and recorded: “mp4_reader.py”/“svo_reader.py”) handle input data streams.
c. Data Processing & Postprocessing:
– Modules in “droid/data_loading”, “droid/data_processing”, “droid/postprocessing”, and “droid/trajectory_utils” manage the ingestion, transformation, and postprocessing of collected data.
d. Evaluation & Training:
– “droid/evaluation” and “droid/training” (plus corresponding scripts in “scripts/evaluation” and “scripts/training”) implement policy evaluation, model training, and sanity-checks.
e. Deployment and Environment Management:
– Distinct Dockerfiles and docker-compose files (in “.docker” and GitHub workflows) show that the platform supports multiple environments (laptop vs. nuc configurations), ensuring reproducibility and ease of deployment.
f. Supporting Utilities & Miscellaneous:
– “droid/misc” and “droid/plotting” include helpful functionality like parameters, visualization, and utility functions crucial for logging, version control, and data analysis.
Relationships and Interactions
• Data flows start from the hardware layer (sensors and controllers) and pass through the data acquisition layer (user interface and camera_utils) toward data processing pipelines.
• Processed data is then used for evaluation and training, with rich postprocessing functionality.
• Additionally, configuration files and Docker/CI configurations indicate that setting up and deploying the system is an integral part.
• There are significant interactions between the modules – for instance, calibration feeds into data acquisition; processed data feeds into evaluation routines; and training modules depend on properly collected and processed data.
Architectural Patterns & Technologies
• The system adopts containerization via Docker and is built to support multiple deployment environments, which speaks to patterns like “Infrastructure as Code”.
• The modular decomposition (with directories clearly split according to functionality such as “camera_utils”, “controllers”, “data_processing”, “robot_ik”, etc.) may suggest a loosely coupled, layered architecture.
• Scripts and CI/CD workflows indicate continuous integration practices.
• The separation of configuration from code (e.g., config/fr3 vs. config/panda) showcases a design favoring parameterization for hardware variability.