How dlr-rm/stable-baselines3 works
Stable-Baselines3 is a PyTorch reinforcement-learning library. A user selects an algorithm and policy, connects an environment, trains through repeated environment interaction and learning updates, then uses the trained model to predict actions. On-policy algorithms collect rollouts; off-policy algorithms store transitions and learn from replay. Shared policies, vectorized environments, buffers, callbacks, logging, and evaluation support these paths. The README documents custom environments and policies, Dict observations, callbacks, and TensorBoard support; the sampled source confirms selected policy, buffer, and vector-environment details, so uncertain internal wiring is omitted.
Main entry points: off_policy_algorithm.py Off-policy loop · base_class.py Model interface · on_policy_algorithm.py On-policy loop · sac.py SAC
Components 23
RL algorithms
Training runtime
- Model interface
stable_baselines3/common/base_class.py - On-policy loop
stable_baselines3/common/on_policy_algorithm.py - Off-policy loop
stable_baselines3/common/off_policy_algorithm.py
Policies and learning
- Shared policies
stable_baselines3/common/policies.py - Algorithm policies
stable_baselines3/sac/policies.py - Rollout buffer
stable_baselines3/common/buffers.py - Replay buffer
stable_baselines3/common/buffers.py - HER replay
stable_baselines3/her/her_replay_buffer.py
Environment tools
- Vector environments
stable_baselines3/common/vec_env/base_vec_env.py - Environment wrappers
stable_baselines3/common/atari_wrappers.py - Example environments
stable_baselines3/common/envs
Monitoring and evaluation
- Callbacks
stable_baselines3/common/callbacks.py - Logger
stable_baselines3/common/logger.py - Evaluation
stable_baselines3/common/evaluation.py
Other components
- RL user
- Gymnasium environment
- TensorBoard
Connections 37
- RL user → A2C: selects and trains
- RL user → PPO: selects and trains
- RL user → DQN: selects and trains
- RL user → DDPG: selects and trains
- RL user → SAC: selects and trains
- RL user → TD3: selects and trains
- A2C → On-policy loop: uses loop
- PPO → On-policy loop: uses loop
- DQN → Off-policy loop: uses loop
- DDPG → Off-policy loop: uses loop
- SAC → Off-policy loop: uses loop
- TD3 → Off-policy loop: uses loop
- A2C → Model interface: extends interface
- PPO → Model interface: extends interface
- DQN → Model interface: extends interface
- DDPG → Model interface: extends interface
- SAC → Model interface: extends interface
- TD3 → Model interface: extends interface
- On-policy loop → Vector environments: steps environments
- Off-policy loop → Vector environments: steps environments
- Vector environments → Gymnasium environment: steps
- Environment wrappers → Gymnasium environment: wraps
- Example environments → Gymnasium environment: implements environments
- On-policy loop → Shared policies: updates policy
- Off-policy loop → Shared policies: updates policy
- SAC → Algorithm policies: selects policy
- TD3 → Algorithm policies: selects policy
- On-policy loop → Rollout buffer: stores rollouts
- Off-policy loop → Replay buffer: stores transitions
- HER replay → Replay buffer: extends replay
- Off-policy loop → HER replay: configures buffer
- On-policy loop → Callbacks: dispatches callbacks
- Off-policy loop → Callbacks: dispatches callbacks
- Model interface → Logger: records metrics
- RL user → Evaluation: requests evaluation
- Evaluation → Vector environments: runs episodes
- Logger → TensorBoard: exports metrics