How nrgrp/saferl-lib works
SafeRL-Lib is a constrained and safe reinforcement-learning toolkit built on Stable-Baselines3 and Safety-Gymnasium. Its principal workflow is to configure an experiment, create a cost-aware environment, train a selected algorithm, and evaluate return, cost, and safety outcomes. Algorithm families share policy, rollout/replay, and training infrastructure; environment wrappers and monitoring support cost-aware operation. The README documents Hydra-driven experiments and evaluation, but the runner’s detailed wiring is not shown in sampled source, so those relationships are cited to the README.
Main entry points: policies.py Actor-critic policies · off_policy_algorithm.py Off-policy engine · main.py Experiment runner · wrappers.py Safety wrappers
Components 24
Experiment flow
- Experiment runner
saferl/examples/main.py - Hydra configs
saferl/examples/configs - Environment configs
saferl/examples/configs/env - Callback configs
saferl/examples/configs/callback
Safe algorithms
Training foundations
- Off-policy engine
saferl/common/off_policy_algorithm.py - On-policy engine
saferl/common/on_policy_algorithm.py - Trust-region engine
saferl/common/trust_region_algorithm.py - Actor-critic policies
saferl/common/policies.py - Cost buffers
saferl/common/buffers.py - Training callbacks
saferl/common/callbacks.py
Environment and evaluation
- Safety wrappers
saferl/common/wrappers.py - Cost monitor
saferl/common/monitor.py - Environment utilities
saferl/common/utils.py - Custom environments
saferl/common/envs - Evaluation utilities
saferl/common/utils.py
Other components
- Researcher
- Safety-Gymnasium
- Stable-Baselines3
Connections 37
- Researcher → Experiment runner: runs experiment
README.md - Experiment runner → Hydra configs: loads settings
README.md - Experiment runner → CSAC-LB: selects algorithm
README.md - Experiment runner → Environment configs: configures environment
README.md - Experiment runner → Callback configs: configures callbacks
README.md - Experiment runner → Evaluation utilities: evaluates model
README.md - CSAC-LB → Off-policy engine: extends
saferl/algorithms/csac_lb/csac_lb.py - CSAC-LB → Cost buffers: uses
saferl/algorithms/csac_lb/csac_lb.py - CSAC-LB → Actor-critic policies: uses
saferl/algorithms/csac_lb/csac_lb.py - CSAC-LB → Safety wrappers: uses
saferl/algorithms/csac_lb/csac_lb.py - SAC-Lag → Off-policy engine: extends
saferl/algorithms/sac_lag/sac_lag.py - SAC-Lag → Cost buffers: uses
saferl/algorithms/sac_lag/sac_lag.py - SAC-Lag → Actor-critic policies: uses
saferl/algorithms/sac_lag/sac_lag.py - SAC-Lag → Safety wrappers: uses
saferl/algorithms/sac_lag/sac_lag.py - WCSAC → Off-policy engine: extends
saferl/algorithms/wcsac/wcsac.py - WCSAC → Cost buffers: uses
saferl/algorithms/wcsac/wcsac.py - WCSAC → Actor-critic policies: uses
saferl/algorithms/wcsac/wcsac.py - WCSAC → Safety wrappers: uses
saferl/algorithms/wcsac/wcsac.py - SAC → Off-policy engine: extends
saferl/algorithms/sac/sac.py - SAC → Cost buffers: uses
saferl/algorithms/sac/sac.py - SAC → Safety wrappers: uses
saferl/algorithms/sac/sac.py - CPO → Trust-region engine: extends
saferl/algorithms/cpo/cpo.py - CPO → Actor-critic policies: uses
saferl/algorithms/cpo/cpo.py - APPO → On-policy engine: extends
saferl/algorithms/appo/appo.py - APPO → Actor-critic policies: uses
saferl/algorithms/appo/appo.py - Off-policy engine → Actor-critic policies: uses
saferl/common/off_policy_algorithm.py - Off-policy engine → Cost buffers: samples
saferl/common/off_policy_algorithm.py - On-policy engine → Actor-critic policies: uses
saferl/common/on_policy_algorithm.py - On-policy engine → Cost buffers: fills rollouts
saferl/common/on_policy_algorithm.py - Trust-region engine → On-policy engine: extends
saferl/common/trust_region_algorithm.py - Trust-region engine → Actor-critic policies: uses
saferl/common/trust_region_algorithm.py - Environment utilities → Safety wrappers: wraps environments
saferl/common/utils.py - Environment utilities → Cost monitor: uses
saferl/common/utils.py - Environment utilities → Custom environments: creates environments
saferl/common/utils.py - Safety wrappers → Safety-Gymnasium: wraps
- Off-policy engine → Stable-Baselines3: extends framework
- On-policy engine → Stable-Baselines3: extends framework