# tanayrs/segwayrl architecture

> Architecture diagram of the GitHub repository tanayrs/segwayrl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tanayrs/segwayrl
- Repository: https://github.com/tanayrs/segwayrl

## Overview

##### Project Type and Purpose
- This is an embedded-reinforcement-learning system for balancing a physical cart-pendulum (Segway). It combines
a) an offline RL training pipeline (simulation, model training/plotting)
b) on-hardware deployment (sensor/actuator interface, controllers, calibration, logging)
c) on-hardware RL fine-tuning.

##### Key Components
A. Offline Training Pipeline
- python/environment: custom CartPole RL environment
- python/models: DQN & PPO training scripts (train_dqn.py, train_ppo.py), plotting/rendering code
- Training/: saved model checkpoints (DQN_model.zip, PPO_model.zip) and training plots

B. On-Hardware Deployment
- Sensors:
- IMU (hardware/main/imu.py)
- Encoder (hardware/main/encoder.py)
- Signal Processing: filters module in hardware/main and calibration modules (deadband_compensation, encoder_characterisation)
- Controllers:
- PID controller (hardware/main/pid_main.py + controller.py)
- RL controller (hardware/main/main.py + controller_rl.py)
- Actuators: motor driver interface (hardware/main/motor.py) and physical DC motor
- Data Logging: hardware/main/logger.py writing to system_logs/ and robot_logs/

C. On-Hardware Fine-Tuning
- hardware/tuning: scripts for continuing RL training on hardware using pre-trained checkpoints

##### Relationships & Data Flows
- Offline: Simulation → train scripts → generate model checkpoints → saved in Training/
- Deployment: hardware/main/main.py loads chosen model → reads sensors → applies filters → controller_rl selects action → motor driver → physical system reacts → sensors produce new data → logger records
- PID Mode: Same loop but controller.py outputs actions based on PID logic
- Calibration Modules: Deadband and encoder_characterisation run offline or pre-deployment to calibrate sensors/actuators; results feed into filters/config
- Fine-Tuning: hardware/tuning scripts interface with hardware/main loop to update model parameters on-device

##### Architectural Patterns & Principles
- Layered separation:
- Layer 1 (Offline Training)
- Layer 2 (Deployment & Calibration Tools)
- Layer 3 (Physical Hardware & Real-Time Loop)
- Modular design: Each function (environment, models, calibration, control, logging) isolated to its own package
- Switchable controller strategy: runtime selection between PID and RL

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Tier A: Offline Training Pipeline
    subgraph "Offline Training Pipeline"
        direction TB
        Sim["Simulation\n(Custom CartPole Env)"]:::offline
        Env["Environment\n(custom_cartpole.py, utils.py)"]:::offline
        DQN["DQN Training\ntrain_dqn.py"]:::offline
        PPO["PPO Training\ntrain_ppo.py"]:::offline
        Plot["Plotting & Rendering\nplotting_code.py, render_models.py"]:::offline
        Checkpoints["Saved Checkpoints\nDQN_model.zip\nPPO_model.zip"]:::storage
        Plots["Training Plots"]:::storage
    end

    Sim -->|"runs on"| Env
    Env -->|"feeds data to"| DQN
    Env -->|"feeds data to"| PPO
    DQN -->|"outputs"| Checkpoints
    PPO -->|"outputs"| Checkpoints
    DQN -->|"generates"| Plots
    PPO -->|"generates"| Plots
    Plot -->|"visualizes"| Plots

    %% Tier B: Deployment & Calibration
    subgraph "Deployment & Calibration Software"
        direction TB
        subgraph Sensors_Module
            direction TB
            IMU["IMU Sensor Interface\nimu.py"]:::deploy
            ENC["Encoder Sensor Interface\nencoder.py"]:::deploy
        end
        subgraph Filters_Module
            direction TB
            Filters["Filters Module\nfilters.py"]:::deploy
        end
        subgraph Calibration_Deadband
            direction TB
            CalDB["Deadband Compensation\n(deadband_compensation/)"]:::deploy
        end
        subgraph Calibration_Encoder
            direction TB
            CalEnc["Encoder Characterisation\n(encoder_characterisation/)"]:::deploy
        end
        subgraph Controllers
            direction TB
            PID["PID Controller\npid_main.py, controller.py"]:::deploy
            RL["RL Controller\nmain.py, controller_rl.py"]:::deploy
        end
        MotorDrv["Motor Driver Interface\nmotor.py, motor_encoder_test.py"]:::deploy
        Logger["Data Logger\nlogger.py"]:::deploy
    end

    Checkpoints -->|"loads model"| RL
    CalDB -->|"provides params"| Filters
    CalEnc -->|"provides params"| Filters
    IMU -->|"raw data"| Filters
    ENC -->|"raw data"| Filters
    Filters -->|"filtered data"| PID
    Filters -->|"filtered data"| RL
    PID -->|"control signal"| MotorDrv
    RL -->|"action signal"| MotorDrv
    MotorDrv -->|"drives"| Plant
    Logger -->|"records to"| SystemLogs
    Logger -->|"records to"| RobotLogs

    %% Tier C: Physical Hardware & Control Loop
    subgraph "Physical Hardware & Control Loop"
        direction TB
        Plant["Physical Plant\nCart-Pendulum"]:::hardware
    end
    Plant -->|"movement"| IMU
    Plant -->|"rotation"| ENC

    %% On-Hardware Fine-Tuning Loop
    subgraph "On-Hardware Fine-Tuning"
        direction TB
        TuningScripts["Tuning Scripts\ncontinue_hardware_training.py,\ntrain_on_hardware.py,\nhardware_env.py,\n..."]:::deploy
        TuningConfig["Config File\nconfig_hardware.yaml"]:::deploy
        HardwareModels["Hardware Trained Models\nfinal_hardware_model.zip\nfinal_hardware_model_PPO.zip"]:::storage
    end

    Plant -->|"runtime data"| TuningScripts
    TuningScripts -->|"updates"| HardwareModels
    HardwareModels -->|"new model"| RL

    %% Storage Logs
    SystemLogs(["System Logs"]):::storage
    RobotLogs(["Robot Logs"]):::storage

    %% Click Events
    click Env "https://github.com/tanayrs/segwayrl/blob/main/python/environment/custom_cartpole.py"
    click Env "https://github.com/tanayrs/segwayrl/blob/main/python/environment/utils.py"
    click DQN "https://github.com/tanayrs/segwayrl/blob/main/python/models/train_dqn.py"
    click PPO "https://github.com/tanayrs/segwayrl/blob/main/python/models/train_ppo.py"
    click Plot "https://github.com/tanayrs/segwayrl/blob/main/python/models/plotting_code.py"
    click Plot "https://github.com/tanayrs/segwayrl/blob/main/python/models/render_models.py"
    click Checkpoints "https://github.com/tanayrs/segwayrl/blob/main/Training/Saved Models/DQN_model.zip"
    click Checkpoints "https://github.com/tanayrs/segwayrl/blob/main/Training/Saved Models/PPO_model.zip"
    click Plots "https://github.com/tanayrs/segwayrl/tree/main/Training/Plots"
    click IMU "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/imu.py"
    click ENC "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/encoder.py"
    click PID "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/pid_main.py"
    click PID "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/controller.py"
    click RL "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/main.py"
    click RL "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/controller_rl.py"
    click MotorDrv "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/motor.py"
    click MotorDrv "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/motor_encoder_test.py"
    click Logger "https://github.com/tanayrs/segwayrl/blob/main/hardware/main/logger.py"
    click CalDB "https://github.com/tanayrs/segwayrl/tree/main/hardware/deadband_compensation/"
    click CalEnc "https://github.com/tanayrs/segwayrl/tree/main/hardware/encoder_characterisation/"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/continue_hardware_training.py"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/train_on_hardware.py"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/hardware_env.py"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/gpio_cleanup.py"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/imu.py"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/motor.py"
    click TuningScripts "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/safety_monitor.py"
    click TuningConfig "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/config_hardware.yaml"
    click HardwareModels "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/hardware_trained_models/final_hardware_model.zip"
    click HardwareModels "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/hardware_trained_models/final_hardware_model_PPO.zip"
    click HardwareModels "https://github.com/tanayrs/segwayrl/blob/main/hardware/tuning/hardware_trained_models/interrupted_model.zip"

    %% Styles
    classDef offline fill:#d0e1f9,stroke:#0366d6,color:#0366d6;
    classDef deploy fill:#d4f7d4,stroke:#28a745,color:#155724;
    classDef hardware fill:#f9d2cb,stroke:#e5533c,color:#e5533c;
    classDef storage fill:#e0e0e0,stroke:#6a737d,color:#6a737d;
```
