ubtech-robot/globalhumanoidrobotchallenge_2026_baseline

1. Overall shape

- This repository is a robotics baseline built on the LeRobot framework, with a custom Ubtech_sim Isaac Sim package plus an extended src/lerobot stack for teleoperation, data collection, training, and policy deployment.

- The architecture is pipeline-oriented: inputs from cameras/teleoperators/robots → processors → datasets/training → policies → runtime inference/control.

2. Simulation layer: Ubtech_sim/

- Ubtech_sim/main.py is the entry into a standalone Isaac Sim simulation workflow.

- SceneBuilder, RobotArticulation, DualArmIK, grasp_planner, and coordinate_utils together define the simulated world, robot kinematics, grasping logic, and coordinate transforms.

- YAML configs such as Conveyor_Sorting.yaml, Foam_Inlaying.yaml, Packing_Box.yaml, and Part_Sorting.yaml indicate task-scene specialization; DataLogger suggests simulation-side logging for later dataset creation or debugging.

3. Framework boundary: src/lerobot/

- src/lerobot is the reusable robotics framework layer. It is not just one application; it contains abstractions for robots, cameras, teleoperators, datasets, policies, RL, and deployment.

- The GHRC baseline uses this framework to standardize how observations, actions, and models are connected across simulation and real/virtual hardware.

4. Robot and actuation abstraction

- src/lerobot/robots/ defines follower robot interfaces and robot-specific adapters for many platforms; the competition-relevant simulation path is centered on Walker S2 support under walker_s2_sim.

- src/lerobot/motors/ is the low-level hardware boundary, with separate implementations for actuator families such as Dynamixel, Feetech, Damiao, and Robstride.

- This layer isolates model-level control from the underlying robot hardware or simulator implementation.

5. Teleoperation boundary

- src/lerobot/teleoperators/ contains leader-device adapters: keyboard, gamepad, phone, leader arms, gloves, and robot-specific teleoperators.

- For the GHRC baseline, the key interaction mode is keyboard teleoperation (walker_s2_keyboard and the generic keyboard teleop modules), which turns human input into normalized robot actions.

- This layer sits upstream of data recording and live control, acting as the human-to-robot input source.

6. Camera and sensor subsystem

- src/lerobot/cameras/ provides a camera abstraction with backends for OpenCV, RealSense, Reachy2, and ZMQ.

- The 4-camera real-time display described in the README is naturally modeled here: camera streams are captured, optionally published over ZMQ, and consumed by teleop, replay, recording, or inference UIs.

- image_server.py and the ZMQ camera backend form the key transport boundary for remote visualization.

7. Data processing and dataset flow

- src/lerobot/processor/ is the normalization/conversion layer between raw robot/camera signals and model-ready tensors.

- It includes processors for observations, actions, tokenization, renaming, slicing, normalization, batching, and robot-policy bridging.

- src/lerobot/datasets/ stores the dataset stack, including the LeRobot dataset implementation, streaming support, image/video utilities, statistics, transforms, and dataset publishing helpers.

- The README’s dataset format requirement maps to this subsystem: recording produces standardized LeRobotDataset V3.0 artifacts, with Hugging Face used for external dataset storage/distribution.

8. Training and policy families

- src/lerobot/policies/ contains multiple model families: ACT, Diffusion, Pi0/Pi0.5, SmolVLA, VQBeT, SARM, TDMPC, RTC, GROOT, XVLA, and others.

- Each policy typically has its own configuration, model, and processor, which means training is structured as: dataset batch → policy-specific processor → model forward/backward → optimizer/scheduler.

- src/lerobot/optim/ and src/lerobot/configs/ provide the training-time control plane for hyperparameters, optimizer selection, and run configuration.

9. Inference and online execution

- src/lerobot/async_inference/ supports a split execution model with a policy server and robot client, enabling policies to run remotely or asynchronously from the robot loop.

- src/lerobot/transport/ contains gRPC service definitions and generated stubs, which are the explicit network boundary for this inference/control split.

- This is the main path for low-latency deployment: observations are sent to the policy process, actions are returned, and the robot client executes them while optionally recording outputs.

10. RL and evaluation tooling

- src/lerobot/rl/ adds a reinforcement-learning-oriented loop with actor/learner components, buffers, queues, evaluation helpers, and WandB integration.

- src/lerobot/envs/ wraps external benchmark environments such as LIBERO and MetaWorld, showing that the framework supports more than just the GHRC humanoid simulation path.

- These modules form a separate training/eval loop from imitation learning, but share the same dataset, processor, and policy infrastructure.

11. Competition-specific Walker S2 path

- The README’s main baseline is the Walker S2 simulation + keyboard teleoperation + 4-camera view + imitation learning workflow.

- In code terms, that path is implemented by the Walker S2 simulator modules under robots/walker_s2_sim, the keyboard teleoperator modules, the camera stack, and the LeRobot dataset/policy subsystems.

- This is the core architecture to draw if focusing on the GHRC 2026 baseline.

12. Entry points and operational tooling

- src/lerobot/scripts/ contains the runnable entry scripts for the major lifecycle steps: setup, calibration, camera discovery, teleoperation, recording, replay, training, evaluation, and dataset editing.

- Dockerfile, run.sh, pyproject.toml, and setup.py indicate a containerized, package-managed runtime intended to make Isaac Sim, data collection, and training reproducible.

- The repo also relies on external assets via the assets/ submodule and Hugging Face-hosted resources for large simulation and dataset files.