# raykr/carla-sb3-rl-training-environment architecture

> Architecture diagram of the GitHub repository raykr/carla-sb3-rl-training-environment, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/raykr/carla-sb3-rl-training-environment
- Repository: https://github.com/raykr/carla-sb3-rl-training-environment

## Overview

##### Project type and purpose
- This is a reinforcement‑learning training and evaluation environment built on the CARLA autonomous‑driving simulator. It uses Stable Baselines 3 algorithms to train agents, wraps CARLA as a Gym‑style environment, collects data (both manually and via RL), and optionally preprocesses observations through a Variational Autoencoder (VAE). It also supports full experiment orchestration, logging (TensorBoard), evaluation, and VAE training.

##### File structure
- Top level:
- config.py: central configuration
- train.py / eval.py / run_experiments.py: entry points for training, evaluation, batch experiments
- utils.py / eval_plots.py: helper functions and plot generation
- requirements.txt
- carla_env/:
- envs/: Gym‑style environments (birdview, route following, data collection)
- navigation/: modular navigation stack (global planner, local planner, agent, controller)
- rewards.py, state_commons.py: reward functions and standard state definitions
- wrappers.py: observation/action wrappers (history buffer, frame skip)
- tools/: HUD, misc utilities for CARLA
- vae/:
- train_vae.py / models.py / utils/: data loaders, training loops for VAE
- log_dir/: saved VAE checkpoints, samples, progress
- tensorboard/: logs, model checkpoints, evaluation CSVs

##### README insights
- Describes configuration parameters (algorithm, state, VAE, reward, wrappers)
- Explains workflows: train, evaluate, run experiments, collect data, train VAE

##### Main components & interactions
A. CARLA Simulator
- Acts as external service (client/server) providing sensors and world feedback
B. Gym‑Style Environment (carla_env/envs/*.py)
- Wraps CARLA API into reset(), step() calls
- Applies wrappers (state builder, history, frame‑skip)
C. State Processing
- Raw sensor data → state_commons (numeric features, waypoint vectors)
- Optional VAE module: raw RGB/seg images → VAE encoder → latent representation
D. Navigation Stack
- GlobalRoutePlanner: plans high‑level route
- LocalPlanner / Controller: converts waypoints into throttle, steer, brake
- Agents (basic_agent, roaming_agent) for data collection or baseline policies
E. Reward Function
- reward.py computes scalar RL rewards based on state, actions, config
F. RL Algorithm (Stable Baselines 3)
- Algorithm class (DDPG, PPO, SAC) configured via config.py
- Policy network consumes processed state, outputs actions
- Logs metrics, checkpoints
G. Experiment Orchestrator
- run_experiments.py: iterates configs, automates train+eval
H. Evaluation Pipeline
- eval.py: loads model, runs fixed routes, logs performance CSV
- eval_plots.py: visualizes evaluation metrics
I. TensorBoard + Checkpoint Storage
- tensorboard/ stores event files, progress.csv, model_zips
J. VAE Training Pipeline
- vae/train_vae.py: loads images via vae/utils/loaders, trains models in vae/models.py, logs to vae/log_dir

##### Architectural patterns and design principles
- Modular separation: CARLA client, env interface, state/reward modules, navigation, RL core, logging
- Config‑driven: single source config.py controls algorithms, states, rewards, wrappers, seeds
- Pipeline pattern: data collection → preprocessing (VAE) → RL training → logging → evaluation
- Extensibility: drop‑in new reward_fn(), wrapper, env, VAE model via directory structure

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry and Config
    subgraph "Entry Points"
        Train["train.py"]:::orchestrator
        Eval["eval.py"]:::orchestrator
        Runner["run_experiments.py"]:::orchestrator
    end

    Config["config.py"]:::config

    %% External Simulator
    CARLA["CARLA Simulator"]:::external

    %% Environment Layer
    subgraph "Environment Layer"
        EnvBird["Bird-view Env"]:::env
        EnvRoute["Route-following Env"]:::env
        EnvCollectManual["Manual Data-collection Env"]:::env
        EnvCollectRL["RL Data-collection Env"]:::env
        Wrappers["Wrappers"]:::env
    end

    %% Navigation Stack
    subgraph "Navigation Stack"
        GlobalPlanner["GlobalRoutePlanner"]:::env
        GlobalDAO["GlobalRoutePlannerDAO"]:::env
        LocalPlanner["LocalPlanner"]:::env
        Controller["Controller"]:::env
        PlannerBase["Planner Base"]:::env
        AgentIface["Generic Agent Interface"]:::env
        BasicAgent["Basic Agent"]:::env
        RoamingAgent["Roaming Agent"]:::env
    end

    %% State and Reward
    subgraph "State & Reward"
        StateCommons["State Commons"]:::state
        Reward["Reward Module"]:::state
    end

    %% VAE Pipeline
    subgraph "VAE Pipeline"
        VAETrain["vae/train_vae.py"]:::vae
        VAEModel["vae/models.py"]:::vae
        VAELoader["vae/utils/loaders.py"]:::vae
        VAELearn["vae/utils/learning.py"]:::vae
        VAEMisc["vae/utils/misc.py"]:::vae
        VAELogs["vae/log_dir/"]:::vae
    end

    %% RL Core and Utils
    RLAlgo["RL Algorithm (SB3)"]:::rl
    Utils["utils.py"]:::rl

    %% Logging
    Tensorboard["TensorBoard Logs"]:::log

    %% Click Events
    click Config "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/config.py"
    click Train "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/train.py"
    click Eval "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/eval.py"
    click Runner "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/run_experiments.py"
    click EnvBird "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/envs/birdview_env.py"
    click EnvRoute "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/envs/carla_route_env.py"
    click EnvCollectManual "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/envs/collect_data_manual_env.py"
    click EnvCollectRL "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/envs/collect_data_rl_env.py"
    click Wrappers "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/wrappers.py"
    click StateCommons "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/state_commons.py"
    click Reward "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/rewards.py"
    click VAETrain "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/vae/train_vae.py"
    click VAEModel "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/vae/models.py"
    click VAELoader "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/vae/utils/loaders.py"
    click VAELearn "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/vae/utils/learning.py"
    click VAEMisc "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/vae/utils/misc.py"
    click GlobalPlanner "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/global_route_planner.py"
    click GlobalDAO "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/global_route_planner_dao.py"
    click LocalPlanner "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/local_planner.py"
    click Controller "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/controller.py"
    click PlannerBase "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/planner.py"
    click AgentIface "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/agent.py"
    click BasicAgent "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/basic_agent.py"
    click RoamingAgent "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/carla_env/navigation/roaming_agent.py"
    click Utils "https://github.com/raykr/carla-sb3-rl-training-environment/blob/main/utils.py"
    click Tensorboard "https://github.com/raykr/carla-sb3-rl-training-environment/tree/main/tensorboard/"

    %% Connections
    Train -->|uses| Config
    Eval -->|uses| Config
    Runner -->|orchestrates| Train
    Runner -->|orchestrates| Eval

    Config -->|parameters| EnvBird
    Config -->|parameters| EnvRoute
    Config -->|parameters| EnvCollectManual
    Config -->|parameters| EnvCollectRL
    Config -->|parameters| Wrappers
    Config -->|parameters| Reward
    Config -->|parameters| RLAlgo

    CARLA -->|sensors/world| EnvBird
    CARLA -->|sensors/world| EnvRoute
    CARLA -->|sensors/world| EnvCollectManual
    CARLA -->|sensors/world| EnvCollectRL

    EnvBird -->|wrapped reset/step| StateCommons
    EnvRoute -->|wrapped reset/step| StateCommons
    EnvCollectManual -->|wrapped reset/step| StateCommons
    EnvCollectRL -->|wrapped reset/step| StateCommons
    Wrappers -->|applies| StateCommons

    StateCommons -->|features| RLAlgo
    StateCommons -->|features| Reward
    Reward -->|"reward signal"| RLAlgo

    %% Optional VAE
    EnvBird -->|"RGB/Seg Frames"| VAETrain
    VAETrain -->|train->Model| VAEModel
    VAELoader -->|loads data| VAETrain
    VAELearn -->|helpers| VAETrain
    VAEMisc -->|utils| VAETrain
    VAETrain -->|"latent"| RLAlgo
    VAETrain -->|logs| VAELogs

    %% Navigation interaction
    RLAlgo -->|waypoints| GlobalPlanner
    GlobalPlanner -->|route| GlobalDAO
    GlobalDAO -->|data| GlobalPlanner
    GlobalPlanner -->|waypoints| LocalPlanner
    LocalPlanner -->|commands| Controller
    PlannerBase -->|interface| GlobalPlanner
    PlannerBase -->|interface| LocalPlanner
    AgentIface -->|implements| BasicAgent
    AgentIface -->|implements| RoamingAgent

    %% RL action loop
    RLAlgo -->|Action Commands| EnvBird
    RLAlgo -->|Action Commands| EnvRoute
    RLAlgo -->|Action Commands| EnvCollectRL

    %% Logging
    RLAlgo -->|Logs/Checkpoints| Tensorboard
    Utils -->|helpers| RLAlgo

    %% Styling
    classDef env fill:#bbdefb,stroke:#0d47a1,color:#0d47a1
    classDef state fill:#c8e6c9,stroke:#2e7d32,color:#2e7d32
    classDef rl fill:#ffcc80,stroke:#ef6c00,color:#e65100
    classDef log fill:#e1bee7,stroke:#4a148c,color:#4a148c
    classDef vae fill:#e0e0e0,stroke:#424242,color:#212121
    classDef config fill:#fff9c4,stroke:#f57f17,color:#f57f17
    classDef orchestrator fill:#ffe082,stroke:#ff8f00,color:#bf360c
    classDef external fill:#90a4ae,stroke:#37474f,color:#263238
```
