# nrgrp/saferl-lib architecture

> Architecture diagram of the GitHub repository nrgrp/saferl-lib, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nrgrp/saferl-lib
- Repository: https://github.com/nrgrp/saferl-lib

## Overview

SafeRL-Lib is a constrained and safe reinforcement-learning toolkit built on Stable-Baselines3 and Safety-Gymnasium. Its principal workflow is to configure an experiment, create a cost-aware environment, train a selected algorithm, and evaluate return, cost, and safety outcomes. Algorithm families share policy, rollout/replay, and training infrastructure; environment wrappers and monitoring support cost-aware operation. The README documents Hydra-driven experiments and evaluation, but the runner’s detailed wiring is not shown in sampled source, so those relationships are cited to the README.

Main entry points: [`saferl/common/policies.py`](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/policies.py) (Actor-critic policies), [`saferl/common/off_policy_algorithm.py`](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/off_policy_algorithm.py) (Off-policy engine), [`saferl/examples/main.py`](https://github.com/nrgrp/saferl-lib/blob/main/saferl/examples/main.py) (Experiment runner), [`saferl/common/wrappers.py`](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/wrappers.py) (Safety wrappers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiment["Experiment flow"]
  node_experiment_runner["Experiment runner<br/>[main.py]"]
  node_experiment_config["Hydra configs"]
  node_environment_configs["Environment configs"]
  node_callback_configs["Callback configs"]
end

subgraph group_algorithms["Safe algorithms"]
  node_csac["CSAC-LB<br/>[csac_lb.py]"]
  node_sac_lag["SAC-Lag<br/>[sac_lag.py]"]
  node_wcsac["WCSAC<br/>[wcsac.py]"]
  node_sac["SAC<br/>[sac.py]"]
  node_cpo["CPO<br/>[cpo.py]"]
  node_appo["APPO<br/>[appo.py]"]
end

subgraph group_training["Training foundations"]
  node_off_policy["Off-policy engine"]
  node_on_policy["On-policy engine"]
  node_trust_region["Trust-region engine"]
  node_policies["Actor-critic policies<br/>[policies.py]"]
  node_buffers[("Cost buffers<br/>[buffers.py]")]
  node_callbacks["Training callbacks<br/>[callbacks.py]"]
end

subgraph group_environment["Environment and evaluation"]
  node_safety_wrapper["Safety wrappers<br/>[wrappers.py]"]
  node_monitor["Cost monitor<br/>[monitor.py]"]
  node_env_factory["Environment utilities<br/>[utils.py]"]
  node_custom_envs["Custom environments"]
  node_evaluator["Evaluation utilities<br/>[utils.py]"]
end

node_researcher(("Researcher"))
node_safety_gymnasium["Safety-Gymnasium"]
node_stable_baselines["Stable-Baselines3"]

node_researcher -->|"runs experiment"| node_experiment_runner
node_experiment_runner -->|"loads settings"| node_experiment_config
node_experiment_runner -->|"selects algorithm"| node_csac
node_experiment_runner -->|"configures environment"| node_environment_configs
node_experiment_runner -->|"configures callbacks"| node_callback_configs
node_experiment_runner -->|"evaluates model"| node_evaluator
node_csac -->|"extends"| node_off_policy
node_csac -->|"uses"| node_buffers
node_csac -->|"uses"| node_policies
node_csac -->|"uses"| node_safety_wrapper
node_sac_lag -->|"extends"| node_off_policy
node_sac_lag -->|"uses"| node_buffers
node_sac_lag -->|"uses"| node_policies
node_sac_lag -->|"uses"| node_safety_wrapper
node_wcsac -->|"extends"| node_off_policy
node_wcsac -->|"uses"| node_buffers
node_wcsac -->|"uses"| node_policies
node_wcsac -->|"uses"| node_safety_wrapper
node_sac -->|"extends"| node_off_policy
node_sac -->|"uses"| node_buffers
node_sac -->|"uses"| node_safety_wrapper
node_cpo -->|"extends"| node_trust_region
node_cpo -->|"uses"| node_policies
node_appo -->|"extends"| node_on_policy
node_appo -->|"uses"| node_policies
node_off_policy -->|"uses"| node_policies
node_off_policy -->|"samples"| node_buffers
node_on_policy -->|"uses"| node_policies
node_on_policy -->|"fills rollouts"| node_buffers
node_trust_region -->|"extends"| node_on_policy
node_trust_region -->|"uses"| node_policies
node_env_factory -->|"wraps environments"| node_safety_wrapper
node_env_factory -->|"uses"| node_monitor
node_env_factory -->|"creates environments"| node_custom_envs
node_safety_wrapper -.->|"wraps"| node_safety_gymnasium
node_off_policy -.->|"extends framework"| node_stable_baselines
node_on_policy -.->|"extends framework"| node_stable_baselines

click node_experiment_runner "https://github.com/nrgrp/saferl-lib/blob/main/saferl/examples/main.py"
click node_experiment_config "https://github.com/nrgrp/saferl-lib/tree/main/saferl/examples/configs"
click node_environment_configs "https://github.com/nrgrp/saferl-lib/tree/main/saferl/examples/configs/env"
click node_callback_configs "https://github.com/nrgrp/saferl-lib/tree/main/saferl/examples/configs/callback"
click node_csac "https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/csac_lb/csac_lb.py"
click node_sac_lag "https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/sac_lag/sac_lag.py"
click node_wcsac "https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/wcsac/wcsac.py"
click node_sac "https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/sac/sac.py"
click node_cpo "https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/cpo/cpo.py"
click node_appo "https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/appo/appo.py"
click node_off_policy "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/off_policy_algorithm.py"
click node_on_policy "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/on_policy_algorithm.py"
click node_trust_region "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/trust_region_algorithm.py"
click node_policies "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/policies.py"
click node_buffers "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/buffers.py"
click node_callbacks "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/callbacks.py"
click node_safety_wrapper "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/wrappers.py"
click node_monitor "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/monitor.py"
click node_env_factory "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/utils.py"
click node_custom_envs "https://github.com/nrgrp/saferl-lib/tree/main/saferl/common/envs"
click node_evaluator "https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_experiment_runner,node_experiment_config,node_environment_configs,node_callback_configs toneBlue
class node_csac,node_sac_lag,node_wcsac,node_sac,node_cpo,node_appo toneAmber
class node_off_policy,node_on_policy,node_trust_region,node_policies,node_buffers,node_callbacks toneMint
class node_safety_wrapper,node_monitor,node_env_factory,node_custom_envs,node_evaluator toneRose
class node_researcher,node_safety_gymnasium,node_stable_baselines toneIndigo
```

## Components

### Experiment flow

- [Experiment runner](https://github.com/nrgrp/saferl-lib/blob/main/saferl/examples/main.py): `saferl/examples/main.py`
- [Hydra configs](https://github.com/nrgrp/saferl-lib/tree/main/saferl/examples/configs): `saferl/examples/configs`
- [Environment configs](https://github.com/nrgrp/saferl-lib/tree/main/saferl/examples/configs/env): `saferl/examples/configs/env`
- [Callback configs](https://github.com/nrgrp/saferl-lib/tree/main/saferl/examples/configs/callback): `saferl/examples/configs/callback`

### Safe algorithms

- [CSAC-LB](https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/csac_lb/csac_lb.py): `saferl/algorithms/csac_lb/csac_lb.py`
- [SAC-Lag](https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/sac_lag/sac_lag.py): `saferl/algorithms/sac_lag/sac_lag.py`
- [WCSAC](https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/wcsac/wcsac.py): `saferl/algorithms/wcsac/wcsac.py`
- [SAC](https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/sac/sac.py): `saferl/algorithms/sac/sac.py`
- [CPO](https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/cpo/cpo.py): `saferl/algorithms/cpo/cpo.py`
- [APPO](https://github.com/nrgrp/saferl-lib/blob/main/saferl/algorithms/appo/appo.py): `saferl/algorithms/appo/appo.py`

### Training foundations

- [Off-policy engine](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/off_policy_algorithm.py): `saferl/common/off_policy_algorithm.py`
- [On-policy engine](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/on_policy_algorithm.py): `saferl/common/on_policy_algorithm.py`
- [Trust-region engine](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/trust_region_algorithm.py): `saferl/common/trust_region_algorithm.py`
- [Actor-critic policies](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/policies.py): `saferl/common/policies.py`
- [Cost buffers](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/buffers.py): `saferl/common/buffers.py`
- [Training callbacks](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/callbacks.py): `saferl/common/callbacks.py`

### Environment and evaluation

- [Safety wrappers](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/wrappers.py): `saferl/common/wrappers.py`
- [Cost monitor](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/monitor.py): `saferl/common/monitor.py`
- [Environment utilities](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/utils.py): `saferl/common/utils.py`
- [Custom environments](https://github.com/nrgrp/saferl-lib/tree/main/saferl/common/envs): `saferl/common/envs`
- [Evaluation utilities](https://github.com/nrgrp/saferl-lib/blob/main/saferl/common/utils.py): `saferl/common/utils.py`

### Other components

- **Researcher**
- **Safety-Gymnasium**
- **Stable-Baselines3**

## Connections

- Researcher → Experiment runner: runs experiment (evidence: `README.md`)
- Experiment runner → Hydra configs: loads settings (evidence: `README.md`)
- Experiment runner → CSAC-LB: selects algorithm (evidence: `README.md`)
- Experiment runner → Environment configs: configures environment (evidence: `README.md`)
- Experiment runner → Callback configs: configures callbacks (evidence: `README.md`)
- Experiment runner → Evaluation utilities: evaluates model (evidence: `README.md`)
- CSAC-LB → Off-policy engine: extends (evidence: `saferl/algorithms/csac_lb/csac_lb.py`)
- CSAC-LB → Cost buffers: uses (evidence: `saferl/algorithms/csac_lb/csac_lb.py`)
- CSAC-LB → Actor-critic policies: uses (evidence: `saferl/algorithms/csac_lb/csac_lb.py`)
- CSAC-LB → Safety wrappers: uses (evidence: `saferl/algorithms/csac_lb/csac_lb.py`)
- SAC-Lag → Off-policy engine: extends (evidence: `saferl/algorithms/sac_lag/sac_lag.py`)
- SAC-Lag → Cost buffers: uses (evidence: `saferl/algorithms/sac_lag/sac_lag.py`)
- SAC-Lag → Actor-critic policies: uses (evidence: `saferl/algorithms/sac_lag/sac_lag.py`)
- SAC-Lag → Safety wrappers: uses (evidence: `saferl/algorithms/sac_lag/sac_lag.py`)
- WCSAC → Off-policy engine: extends (evidence: `saferl/algorithms/wcsac/wcsac.py`)
- WCSAC → Cost buffers: uses (evidence: `saferl/algorithms/wcsac/wcsac.py`)
- WCSAC → Actor-critic policies: uses (evidence: `saferl/algorithms/wcsac/wcsac.py`)
- WCSAC → Safety wrappers: uses (evidence: `saferl/algorithms/wcsac/wcsac.py`)
- SAC → Off-policy engine: extends (evidence: `saferl/algorithms/sac/sac.py`)
- SAC → Cost buffers: uses (evidence: `saferl/algorithms/sac/sac.py`)
- SAC → Safety wrappers: uses (evidence: `saferl/algorithms/sac/sac.py`)
- CPO → Trust-region engine: extends (evidence: `saferl/algorithms/cpo/cpo.py`)
- CPO → Actor-critic policies: uses (evidence: `saferl/algorithms/cpo/cpo.py`)
- APPO → On-policy engine: extends (evidence: `saferl/algorithms/appo/appo.py`)
- APPO → Actor-critic policies: uses (evidence: `saferl/algorithms/appo/appo.py`)
- Off-policy engine → Actor-critic policies: uses (evidence: `saferl/common/off_policy_algorithm.py`)
- Off-policy engine → Cost buffers: samples (evidence: `saferl/common/off_policy_algorithm.py`)
- On-policy engine → Actor-critic policies: uses (evidence: `saferl/common/on_policy_algorithm.py`)
- On-policy engine → Cost buffers: fills rollouts (evidence: `saferl/common/on_policy_algorithm.py`)
- Trust-region engine → On-policy engine: extends (evidence: `saferl/common/trust_region_algorithm.py`)
- Trust-region engine → Actor-critic policies: uses (evidence: `saferl/common/trust_region_algorithm.py`)
- Environment utilities → Safety wrappers: wraps environments (evidence: `saferl/common/utils.py`)
- Environment utilities → Cost monitor: uses (evidence: `saferl/common/utils.py`)
- Environment utilities → Custom environments: creates environments (evidence: `saferl/common/utils.py`)
- Safety wrappers → Safety-Gymnasium: wraps
- Off-policy engine → Stable-Baselines3: extends framework
- On-policy engine → Stable-Baselines3: extends framework
