How ailiveco/local-trainings works
Main Components
a) Local Training Environment
• PPO Agent (Stable-Baselines3)
• Gym Environment (Humanoid-v5)
• TensorBoard Logger
• Local Storage System
b) Cloud Infrastructure
• Remote Storage Server
• Pre-signed URL Service
• API Authentication Layer
Component Relationships
• Logging: PPO Agent → TensorBoard Logger
• Model Saving: PPO Agent → Local Storage
• Cloud Upload: Local Storage → Remote Storage Server
• Authentication: Local Training Environment ↔ API Authentication Layer
Specific Implementation Details
• Green for active training components (PPO Agent, Gym Environment)
• Blue for storage components (Local Storage, Remote Storage)
• Yellow for monitoring components (TensorBoard)
• Red for security components (API Authentication)
Key Processes
a) Training Loop:
• Initialize PPO → Load Latest Model (if exists) → Train → Save Checkpoint → Upload to Cloud
b) Model Management:
• Save Model → Get Presigned URL → Upload Model → Confirm Upload
Technical Specifications
Annotations showing:
• Python 3.7+ runtime
• Stable-Baselines3 framework
• PPO algorithm configuration
• Save interval (500,000 steps)
• Total timesteps (10M)
Session Management
Session directory structure:
Sessions/
• Agent_Name/
• Skill_Name/
• Tensorboard/
• Models/
The model versioning and recovery system
This will give other engineers a clear understanding of both the system's architecture and its operational flow.