How intelligent-soft-robots/learning_table_tennis_from_scratch works
Main Components
• PAM Robot Interface: This external component interacts with the PAM robot software and manages the low-level hardware or simulation commands (using external config files such as the pam_config).
• Simulation Environment (Mujoco Instances): There are at least two instances:
– The Pseudo-real/Simulated Robot: Controlled by executables (e.g., hysr_one_ball_rl, hysr_one_ball_swing) and reset/reconfiguration commands.
– Extra Balls Module: An optional third Mujoco simulation, used when “extra balls” are enabled.
• RL Learning and Control Module: This part (located in “learning_table_tennis_from_scratch” and extended via hp_optim) implements RL algorithms (using modified baseline PPO modules and possibly SAC). It consumes sensor/state data from the simulations and sends appropriate control signals.
• Configuration Repository: A set of JSON configuration files (hysr_config, reward_config, rl_config, etc.) that dictate simulation parameters, control frequencies, graphical settings and extra ball options.
• External Services & Monitoring: Components include executables for frequency monitoring (hysr_episode_frequencies, hysr_step_frequencies) and visualization (hysr_visualization) as well as integration with Tensorboard.
Relationships and Interactions
• The initialization begins with starting the robots via the “hysr_start_robots” executable, which instantiates the Mujoco simulation environments.
• The simulation environments mirror a “pseudo-real” behavior with one instance simulating the robot and another (if enabled) handling extra ball physics.
• Control commands for executing swings, reset behaviors, or RL-based decisions are dispatched by dedicated executables (residing in the “bin” folder) that reference the appropriate JSON files from “config”.
• The RL learning module, working via a gym environment interface imported from “learning_table_tennis_from_scratch”, interacts bidirectionally with the simulation: sending pressure control commands to the pseudo-real robot and receiving state/reward signals.
• The hyperparameter optimization module (in “hp_optim”) sits alongside the main learning code to adjust algorithm parameters and run experiments.
• Data flow and monitoring are supported through logs (Tensorboard) and frequency monitoring executables.
Architectural Patterns and Technologies
• A layered architecture separating the “control/learning” layer from the “simulation” layer.
• Use of JSON-based configuration for decoupling settings from code execution.
• A modular design that allows switching between modes (debugging, testing, learning) by executing different “hysr_*” binaries.
• Integration with external PAM software and Mujoco simulations.
• An underlying event-based or step-wise control loop is inferred from parameters such as “nb_steps_per_episode”, “algo_time_step” and frequency monitoring.