How 2allen/carla-rl-project works
Project type and purpose
• This is a reinforcement-learning (RL) based autonomous-driving research project built on the CARLA simulator.
• Its goal is to train and evaluate a PPO (Proximal Policy Optimization) agent to drive a virtual car in CARLA.
File structure
• Top-level scripts:
• train.py – orchestrates the training loop, checkpoints, tensorboard logging.
• test.py – loads a saved model, runs evaluation episodes, optionally renders or records video.
• src/ directory:
• environment.py – wraps the CARLA simulator as a Gymnasium-style environment.
• agent.py – implements the RL agent (policy, learning update) using Stable-Baselines3’s PPO.
• utils.py – shared helpers (e.g., video recorder, seeding, logging utilities).
• requirements.txt – lists Python and ML dependencies.
• test.py & train.py integrate src/ components and external services (CARLA server, Tensorboard).
Extract architecture insights from the README
• Core loop: environment ↔ agent ↔ training engine.
• External services:
• CARLA Simulator (Town04, vehicle models, rendering/video).
• Tensorboard for monitoring.
• File system for checkpoint & video storage.
• Data flows: observations, actions, rewards, logs, checkpoints, recorded videos.
Main components
a. CARLA Simulator Server
• Hosts the virtual world, physics, sensors.
• Exposes a socket/API to environment.py.
b. Environment Wrapper (environment.py)
• Translates CARLA data → Gym observations.
• Sends actions → CARLA control API.
• Computes reward.
c. RL Agent (agent.py)
• Implements PPO policy/value networks.
• Receives observations, outputs actions.
• Performs gradient updates.
d. Training Orchestrator (train.py)
• Instantiates agent & environment.
• Runs episodes, collects trajectories.
• Saves checkpoints every N steps.
• Logs metrics to Tensorboard.
e. Evaluation/Test Harness (test.py)
• Loads checkpoints.
• Runs episodes in environment.
• Optionally renders or records video → ./videos.
• Collects evaluation metrics.
f. Auxiliary services
• Tensorboard server (reads logs folder).
• File system (stores logs, checkpoints, videos).
Relationships and interactions
• Agent pulls observations from the wrapper, pushes actions back.
• Training Orchestrator links agent ↔ environment in a loop.
• Orchestrator writes logs → Tensorboard and saves model → disk.
• Test Harness reads model from disk, interacts same as training loop.
• Video Recorder (in utils.py) taps into the environment wrapper when render=False.
Architectural patterns and design principles
• Separation of concerns:
• Simulation (CARLA) vs. RL logic (agent, training).
• Core RL loop vs. evaluation loop.
• Modular wrappers (Gym API) to keep agent agnostic of simulator internals.
• Pluggable components: you could swap PPO for another algorithm by replacing agent.py.
• Clear data-flow channels for reproducibility (deterministic seeding, checkpointing).
Checklist for the engineer
• Clear component labels.
• Technologies: CARLA, Stable-Baselines3, Gymnasium, Tensorboard.