# intelligent-soft-robots/learning_table_tennis_from_scratch architecture

> Architecture diagram of the GitHub repository intelligent-soft-robots/learning_table_tennis_from_scratch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/intelligent-soft-robots/learning_table_tennis_from_scratch
- Repository: https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch

## Overview

##### Main Components
- PAM Robot Interface: This external component interacts with the PAM robot software and manages the low-level hardware or simulation commands (using external config files such as the pam_config).
- Simulation Environment (Mujoco Instances): There are at least two instances:
– The Pseudo-real/Simulated Robot: Controlled by executables (e.g., hysr_one_ball_rl, hysr_one_ball_swing) and reset/reconfiguration commands.
– Extra Balls Module: An optional third Mujoco simulation, used when “extra balls” are enabled.
- RL Learning and Control Module: This part (located in “learning_table_tennis_from_scratch” and extended via hp_optim) implements RL algorithms (using modified baseline PPO modules and possibly SAC). It consumes sensor/state data from the simulations and sends appropriate control signals.
- Configuration Repository: A set of JSON configuration files (hysr_config, reward_config, rl_config, etc.) that dictate simulation parameters, control frequencies, graphical settings and extra ball options.
- External Services & Monitoring: Components include executables for frequency monitoring (hysr_episode_frequencies, hysr_step_frequencies) and visualization (hysr_visualization) as well as integration with Tensorboard.

##### Relationships and Interactions
- The initialization begins with starting the robots via the “hysr_start_robots” executable, which instantiates the Mujoco simulation environments.
- The simulation environments mirror a “pseudo-real” behavior with one instance simulating the robot and another (if enabled) handling extra ball physics.
- Control commands for executing swings, reset behaviors, or RL-based decisions are dispatched by dedicated executables (residing in the “bin” folder) that reference the appropriate JSON files from “config”.
- The RL learning module, working via a gym environment interface imported from “learning_table_tennis_from_scratch”, interacts bidirectionally with the simulation: sending pressure control commands to the pseudo-real robot and receiving state/reward signals.
- The hyperparameter optimization module (in “hp_optim”) sits alongside the main learning code to adjust algorithm parameters and run experiments.
- Data flow and monitoring are supported through logs (Tensorboard) and frequency monitoring executables.

##### Architectural Patterns and Technologies
- A layered architecture separating the “control/learning” layer from the “simulation” layer.
- Use of JSON-based configuration for decoupling settings from code execution.
- A modular design that allows switching between modes (debugging, testing, learning) by executing different “hysr_*” binaries.
- Integration with external PAM software and Mujoco simulations.
- An underlying event-based or step-wise control loop is inferred from parameters such as “nb_steps_per_episode”, “algo_time_step” and frequency monitoring.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Initialization
    S("Start Robots"):::init
    click S "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_start_robots"

    %% Simulation Layer
    subgraph "Simulation Layer"
        subgraph "Primary Simulation Environment"
            RLR("hysr_one_ball_rl"):::sim
            click RLR "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_one_ball_rl"
            RLS("hysr_one_ball_swing"):::sim
            click RLS "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_one_ball_swing"
            SED("Sim Environment Definition"):::sim
            click SED "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/learning_table_tennis_from_scratch/hysr_one_ball_env.py"
        end
        subgraph "Extra Balls Module"
            EBE("Extra Balls Executor"):::sim
            click EBE "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_extra_balls"
            DEB("Display Extra Balls"):::sim
            click DEB "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_display_extra_balls"
            LEB("Log Extra Balls"):::sim
            click LEB "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_log_extra_balls"
        end
    end

    %% External PAM Robot Interface
    PAPI("PAM Robot Interface"):::ext
    click PAPI "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/learning_table_tennis_from_scratch/robot_integrity.py"

    %% RL Learning & Control Layer
    subgraph "RL Learning & Control"
        subgraph "RL Learning Module"
            PPO("PPO Module"):::control
            click PPO "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/learning_table_tennis_from_scratch/modified_baselines_ppo2.py"
            RM("Rewards Module"):::control
            click RM "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/learning_table_tennis_from_scratch/rewards.py"
            RC("RL Config"):::control
            click RC "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/learning_table_tennis_from_scratch/rl_config.py"
        end
    end

    %% Configuration Repository
    subgraph "Configuration Repository"
        HOB("hysr_one_ball_default.json"):::config
        click HOB "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/config/hysr_one_ball_default.json"
        RD("reward_default.json"):::config
        click RD "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/config/reward_default.json"
        OPP("openai_ppo_default.json"):::config
        click OPP "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/config/openai_ppo_default.json"
    end

    %% External Services & Monitoring
    subgraph "Monitoring & Visualization"
        EF("Episode Frequencies"):::ext
        click EF "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_episode_frequencies"
        SF("Step Frequencies"):::ext
        click SF "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_step_frequencies"
        VIS("Visualization"):::ext
        click VIS "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/tree/master/bin/hysr_visualization"
    end

    %% Hyperparameter Optimization Module
    subgraph "Hyperparameter Optimization"
        HPR("HP Optim Run"):::hp
        click HPR "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/hp_optim/run_learning.py"
        HPC("HP Optim Config"):::hp
        click HPC "https://github.com/intelligent-soft-robots/learning_table_tennis_from_scratch/blob/master/hp_optim/hp_optim_ppo.json"
    end

    %% Connections
    S -->|"initiate"| RLR
    S -->|"initiate"| RLS
    S -->|"initiate"| EBE
    S -->|"initiate"| PAPI

    %% RL commands to simulation and PAM interface
    PPO -->|"control"| RLR
    PPO -->|"control"| RLS
    RM -->|"signal"| RLR
    RC -->|"config"| RD

    %% Simulation feedback to RL module
    RLR -->|"state/reward"| PPO
    RLS -->|"state/reward"| RM

    %% Configuration feeds
    HOB ---|"config"| RLR
    OPP ---|"config"| RC
    RD ---|"config"| RM
    HOB ---|"config"| EBE

    %% Monitoring connections
    RLR -->|"log"| EF
    RLS -->|"log"| SF
    EBE ---|"optional_log"| VIS

    %% Hyperparameter Optimization connections
    HPC --> HPR
    HPR -->|"adjust"| PPO

    %% PAM Robot Interface communication
    PPO -->|"commands"| PAPI
    PAPI -->|"feedback"| RM

    %% Styling
    classDef init fill:#e8f5e9,stroke:#43a047,stroke-width:2px;
    classDef sim fill:#fde2e4,stroke:#d62828,stroke-width:2px;
    classDef control fill:#d1e8ff,stroke:#1a5582,stroke-width:2px;
    classDef config fill:#e0f7fa,stroke:#00796b,stroke-width:2px;
    classDef ext fill:#fff3e0,stroke:#fb8c00,stroke-width:2px;
    classDef hp fill:#f3e5f5,stroke:#8e24aa,stroke-width:2px;
```
