How quantumiracle/popular-rl-algorithms works
This repository is a research collection of model-free reinforcement-learning implementations, not a unified application or library. Its common user workflow is to run an algorithm script against an environment, collect experience, update a policy or value model, and evaluate behavior. The README documents additional algorithm families whose individual wiring was not sampled; those are shown as separate capability branches rather than an invented shared pipeline. Source excerpts confirm SAC components, shared network and buffer utilities, and the CEM rollout loop.
Main entry points: sac_v2.py SAC v2 · cem.py Cross-entropy method · sdt_ppo_gae_discrete.py Tree-based PPO · _policies.py Linear policies
Components 21
Off-policy RL
- SAC v2
sac_v2.py - Discrete SAC
sac_discrete.py - DDPG
ddpg.py - TD3
td3.py - DQN
dqn.py - Recurrent policies
rdpg.py
On-policy RL
- PPO variants
ppo_continuous.py - Actor-Critic
ac.py
Specialized methods
- QMIX
qmix.py - Mixture policies
pmoe_sac.py - Tree-based PPO
sdt_ppo_gae_discrete.py - Soft decision tree
SDT/SDT.py - Cross-entropy method
cem/cem.py - QT-Opt
qt_opt_v3.py - Linear policies
cem/_policies.py
Shared components
- Replay buffers
common/buffers.py - Policy/value networks
common/policy_networks.py - Network initialization
common/initialize.py
Other components
- Experimenter
- Gym environments
- Reacher environment
reacher.py
Connections 22
- Experimenter → SAC v2: runs training
- Experimenter → PPO variants: runs training
- Experimenter → Cross-entropy method: runs experiment
- SAC v2 → Gym environments: steps environment
- SAC v2 → Replay buffers: stores and samples
- SAC v2 → Policy/value networks: uses networks
- Policy/value networks → Network initialization: imports initialization
- Experimenter → Reacher environment: selects environment
- Cross-entropy method → Gym environments: evaluates rollouts
- Cross-entropy method → Linear policies: creates policy
- Linear policies → Gym environments: selects actions
- Experimenter → Discrete SAC: runs algorithm
- Experimenter → DDPG: runs algorithm
- Experimenter → TD3: runs algorithm
- Experimenter → DQN: runs algorithm
- Experimenter → Recurrent policies: runs algorithm
- Experimenter → Actor-Critic: runs algorithm
- Experimenter → QMIX: runs algorithm
- Experimenter → Mixture policies: runs algorithm
- Experimenter → Tree-based PPO: runs algorithm
- Tree-based PPO → Soft decision tree: uses tree model
- Experimenter → QT-Opt: runs algorithm