How tanayrs/segwayrl works
Project Type and Purpose
• This is an embedded-reinforcement-learning system for balancing a physical cart-pendulum (Segway). It combines
a) an offline RL training pipeline (simulation, model training/plotting)
b) on-hardware deployment (sensor/actuator interface, controllers, calibration, logging)
c) on-hardware RL fine-tuning.
Key Components
A. Offline Training Pipeline
• python/environment: custom CartPole RL environment
• python/models: DQN & PPO training scripts (train_dqn.py, train_ppo.py), plotting/rendering code
• Training/: saved model checkpoints (DQN_model.zip, PPO_model.zip) and training plots
B. On-Hardware Deployment
• Sensors:
• IMU (hardware/main/imu.py)
• Encoder (hardware/main/encoder.py)
• Signal Processing: filters module in hardware/main and calibration modules (deadband_compensation, encoder_characterisation)
• Controllers:
• PID controller (hardware/main/pid_main.py + controller.py)
• RL controller (hardware/main/main.py + controller_rl.py)
• Actuators: motor driver interface (hardware/main/motor.py) and physical DC motor
• Data Logging: hardware/main/logger.py writing to system_logs/ and robot_logs/
C. On-Hardware Fine-Tuning
• hardware/tuning: scripts for continuing RL training on hardware using pre-trained checkpoints
Relationships & Data Flows
• Offline: Simulation → train scripts → generate model checkpoints → saved in Training/
• Deployment: hardware/main/main.py loads chosen model → reads sensors → applies filters → controller_rl selects action → motor driver → physical system reacts → sensors produce new data → logger records
• PID Mode: Same loop but controller.py outputs actions based on PID logic
• Calibration Modules: Deadband and encoder_characterisation run offline or pre-deployment to calibrate sensors/actuators; results feed into filters/config
• Fine-Tuning: hardware/tuning scripts interface with hardware/main loop to update model parameters on-device
Architectural Patterns & Principles
• Layered separation:
• Layer 1 (Offline Training)
• Layer 2 (Deployment & Calibration Tools)
• Layer 3 (Physical Hardware & Real-Time Loop)
• Modular design: Each function (environment, models, calibration, control, logging) isolated to its own package
• Switchable controller strategy: runtime selection between PID and RL