How robochallenge/robochallengeinference works
1. What this repo is for
- This is an inference/runtime package for RoboChallenge robotics tasks, not a training repo.
- The core job is to connect a policy implementation to a robot API, with a mock robot server for local debugging.
2. Main execution entrypoints
- test.py is the primary test runner used during local evaluation.
- demo.py is the place to plug in custom inference logic; the README points to it as the file to modify for your policy.
- These scripts sit above the rest of the repo and orchestrate loading data, querying robot state, and sending actions.
3. Robot interface boundary
- robot/interface_client.py is the client-facing boundary to the robot’s direct HTTP API.
- The README defines the key endpoints this layer targets: clock-sync, state.pkl, and action.
- This layer is responsible for adapting model output into the robot’s required action shape and timing.
4. Job/runtime orchestration
- robot/job_worker.py appears to manage the execution loop for a job: polling state, invoking inference, and pushing actions.
- Architecturally, this is the control-plane piece that sits between the evaluation harness and the robot interface.
- It likely handles queueing, timing, and the “pending actions” style backpressure exposed by the API.
5. Mock robot subsystem
- mock_server/mock_robot_server.py provides a local stand-in for the real robot service.
- mock_server/mock_rc_robot.py likely implements the simulated robot behavior and state transitions.
- mock_server/mock_settings.py is a key configuration boundary: it selects the active ROBOT_TAG and RECORD_DATA_DIR.
- mock_server/utils.py supports the mock environment with helper functions.
6. How local testing works
- The intended local loop is: start the mock server, then run test.py against it.
- test.py consumes the mock API just like the real interface, so model behavior can be validated without hardware access.
- This means the mock server is the main architectural substitute for the production robot during development.
7. Dataset / episode storage
- The 20250919/ tree contains task-specific recorded episodes used for evaluation or replay.
- Each task folder, such as arrange_paper_cups, pour_fries_into_plate, press_three_buttons, and shred_scrap_paper, contains one or more episode_* directories.
- Each episode is organized into meta/, states/, and videos/, making the dataset a multimodal bundle of metadata, time-series robot state, and camera recordings.
8. Episode file structure
- meta/episode_meta.json stores episode-level metadata.
- states/*.jsonl stores time-ordered robot state traces; some tasks use one stream (states.jsonl), others split per arm (left_states.jsonl, right_states.jsonl).
- videos/*.mp4 contains synchronized camera feeds, with camera names indicating viewpoint and robot type.
9. Observation and action flow
- The robot API returns a pickle payload from state.pkl containing robot state, timestamp, current action, pending action count, and selected camera images.
- The policy consumes these observations and emits a batch of actions through POST /action.
- The data flow is therefore: camera/state observation in, batched control commands out, with the robot server enforcing shape and control-mode consistency.
10. Robot-type dependent control contract
- The repository is explicitly built around multiple robot families: Aloha, Arx5, Ur5, and Franka.
- Action shape and camera availability depend on the robot tag and the chosen action_type (joint vs pos, and left/right variants).
- This is an important architectural boundary: the inference code must be aware of robot-specific action semantics, not just task semantics.
11. Shared utilities
- utils/enums.py, utils/log.py, and utils/util.py hold cross-cutting support for the rest of the system.
- These modules are the common infrastructure for type/enum definitions, logging, and helper logic used by both the robot client and mock server.
12. Packaging and runtime assumptions
- requirements.txt indicates a Python-based runtime.
- The repo is built for a single-process local development workflow but maps cleanly onto a remote robot service in production.
- The main architectural boundary is between the policy/runtime code in this repo and the external robot service defined by the README API.
Main entry points: mock_robot_server.py Mock server · 20250919 Episodes · interface_client.py Interface client · job_worker.py Job worker
Components 17
Entry points
Scripts that launch local inference and evaluation loops.
Robot runtime
Client boundary and job orchestration between policy code and robot service.
- Job worker · Coordinates state polling, inference calls, and action submission.
robot/job_worker.py - Interface client · HTTP boundary for robot state, clock sync, and action submission.
robot/interface_client.py - Robot API · Production robot service exposing state and action endpoints.
Mock robot
Local stand-in for the production robot API used during development.
- Mock server · Local HTTP server that emulates the robot interface.
mock_server/mock_robot_server.py - Mock robot · Simulated robot state and behavior used by the mock server.
mock_server/mock_rc_robot.py - Mock settings · Selects robot tag and recording directory for the mock environment.
mock_server/mock_settings.py - Mock utils · Support functions for the mock robot environment.
mock_server/utils.py
Shared utils
Cross-cutting enums, logging, and helper utilities.
- Enums · Shared constants and type-like definitions.
utils/enums.py - Logging · Common logging setup used across runtime and server code.
utils/log.py - Util helpers · General-purpose helper functions.
utils/util.py
Recorded data
Task episodes used for evaluation, replay, and debugging.
- Episodes · Per-task multimodal episode bundles with metadata, states, and videos.
20250919 - Arrange cups · Recorded episode data for arrange_paper_cups.
20250919/arrange_paper_cups - Pour fries · Recorded episode data for pour_fries_into_plate.
20250919/pour_fries_into_plate - Press buttons · Recorded episode data for press_three_buttons.
20250919/press_three_buttons - Shred paper · Recorded episode data for shred_scrap_paper.
20250919/shred_scrap_paper
Connections 18
- test.py → Job worker: runs · Local evaluation entrypoint launches the job execution loop.
- demo.py → Job worker: implements policy for · Custom inference logic is plugged into the runtime loop.
- Job worker → Interface client: sends actions · Execution loop pushes observations and actions through the robot boundary.
- Interface client → Robot API: calls · Client talks to the production robot HTTP API.
- test.py → Mock server: targets · Local testing is commonly run against the mock robot server.
- Mock server → Mock robot: drives · HTTP server delegates robot behavior and state transitions.
- Mock server → Mock settings: reads config · Mock server is configured by robot tag and recording directory.
- Mock server → Mock utils: uses · Server relies on support helpers for the emulated environment.
- Job worker → demo.py: invokes · Runtime calls the policy hook to produce actions.
- Interface client → Enums: uses · Robot client shares action/state definitions with the rest of the repo.
- Interface client → Util helpers: uses · Client relies on common helper functions.
- Mock server → Logging: logs with · Mock server uses shared logging infrastructure.
- Mock server → Util helpers: uses · Mock server uses common helper functions.
- Mock robot → Enums: uses · Simulated robot follows shared control and state conventions.
- Episodes → Arrange cups: contains · Dataset root organizes task-specific episode collections.
- Episodes → Pour fries: contains · Dataset root organizes task-specific episode collections.
- Episodes → Press buttons: contains · Dataset root organizes task-specific episode collections.
- Episodes → Shred paper: contains · Dataset root organizes task-specific episode collections.