How raykr/carla-sb3-rl-training-environment works
Project type and purpose
• This is a reinforcement‑learning training and evaluation environment built on the CARLA autonomous‑driving simulator. It uses Stable Baselines 3 algorithms to train agents, wraps CARLA as a Gym‑style environment, collects data (both manually and via RL), and optionally preprocesses observations through a Variational Autoencoder (VAE). It also supports full experiment orchestration, logging (TensorBoard), evaluation, and VAE training.
File structure
• Top level:
• config.py: central configuration
• train.py / eval.py / run_experiments.py: entry points for training, evaluation, batch experiments
• utils.py / eval_plots.py: helper functions and plot generation
• requirements.txt
• carla_env/:
• envs/: Gym‑style environments (birdview, route following, data collection)
• navigation/: modular navigation stack (global planner, local planner, agent, controller)
• rewards.py, state_commons.py: reward functions and standard state definitions
• wrappers.py: observation/action wrappers (history buffer, frame skip)
• tools/: HUD, misc utilities for CARLA
• vae/:
• train_vae.py / models.py / utils/: data loaders, training loops for VAE
• log_dir/: saved VAE checkpoints, samples, progress
• tensorboard/: logs, model checkpoints, evaluation CSVs
README insights
• Describes configuration parameters (algorithm, state, VAE, reward, wrappers)
• Explains workflows: train, evaluate, run experiments, collect data, train VAE
Main components & interactions
A. CARLA Simulator
• Acts as external service (client/server) providing sensors and world feedback
B. Gym‑Style Environment (carla_env/envs/*.py)
• Wraps CARLA API into reset(), step() calls
• Applies wrappers (state builder, history, frame‑skip)
C. State Processing
• Raw sensor data → state_commons (numeric features, waypoint vectors)
• Optional VAE module: raw RGB/seg images → VAE encoder → latent representation
D. Navigation Stack
• GlobalRoutePlanner: plans high‑level route
• LocalPlanner / Controller: converts waypoints into throttle, steer, brake
• Agents (basic_agent, roaming_agent) for data collection or baseline policies
E. Reward Function
• reward.py computes scalar RL rewards based on state, actions, config
F. RL Algorithm (Stable Baselines 3)
• Algorithm class (DDPG, PPO, SAC) configured via config.py
• Policy network consumes processed state, outputs actions
• Logs metrics, checkpoints
G. Experiment Orchestrator
• run_experiments.py: iterates configs, automates train+eval
H. Evaluation Pipeline
• eval.py: loads model, runs fixed routes, logs performance CSV
• eval_plots.py: visualizes evaluation metrics
I. TensorBoard + Checkpoint Storage
• tensorboard/ stores event files, progress.csv, model_zips
J. VAE Training Pipeline
• vae/train_vae.py: loads images via vae/utils/loaders, trains models in vae/models.py, logs to vae/log_dir
Architectural patterns and design principles
• Modular separation: CARLA client, env interface, state/reward modules, navigation, RL core, logging
• Config‑driven: single source config.py controls algorithms, states, rewards, wrappers, seeds
• Pipeline pattern: data collection → preprocessing (VAE) → RL training → logging → evaluation
• Extensibility: drop‑in new reward_fn(), wrapper, env, VAE model via directory structure