How thu-ml/tianshou works
Tianshou is a PyTorch reinforcement-learning library with procedural and high-level APIs. The principal workflow configures an algorithm and environment, collects interaction data, stores experience, trains the algorithm, and reports results. Its algorithm scope includes model-free, imitation, model-based, and multi-agent RL; it also supports vectorized environments, prioritized and hindsight replay, and experiment evaluation. The graph combines README-documented capabilities with sampled trainer, collector, environment, and experiment code; links for unsampled subsystem internals are limited to documented workflow relationships.
Main entry points: algorithm_base.py Algorithm interface · collector.py Experience collection · trainer.py Training loop · experiment.py Experiment setup
Components 18
Experiment APIs
- Experiment setup
tianshou/highlevel/experiment.py
Training Runtime
- Training loop
tianshou/trainer.py - Experience collection
tianshou/data/collector.py
Learning Algorithms
- Algorithm interface
tianshou/algorithm/algorithm_base.py - Model-free RL
tianshou/algorithm/modelfree - Imitation RL
tianshou/algorithm/imitation - Model-based RL
tianshou/algorithm/modelbased - Multi-agent RL
tianshou/algorithm/multiagent - Actor and critic nets
tianshou/utils/net
Environment and Data
- Vector environments
tianshou/env/venvs.py - Environment workers
tianshou/env/worker/worker_base.py - Gymnasium environments
- Replay buffers
tianshou/data/buffer - Batch data
tianshou/data/batch.py
Results
- Experiment logging
tianshou/highlevel/logger.py - RL evaluation
tianshou/evaluation/rliable_evaluation.py - TensorBoard and W&B
tianshou/utils/logger
Other components
- Researcher or app
Connections 21
- Researcher or app → Experiment setup: configures experiment
- Researcher or app → Algorithm interface: uses algorithm API
- Experiment setup → Training loop: runs training
- Experiment setup → Vector environments: creates environments
- Training loop → Experience collection: collects experience
- Training loop → Algorithm interface: requests updates
- Algorithm interface → Model-free RL: implements
- Algorithm interface → Imitation RL: implements
- Algorithm interface → Model-based RL: implements
- Algorithm interface → Multi-agent RL: implements
- Model-free RL → Actor and critic nets: uses networks
- Experience collection → Algorithm interface: gets actions
- Experience collection → Vector environments: steps environments
- Vector environments → Environment workers: dispatches work
- Environment workers → Gymnasium environments: interacts with
- Experience collection → Replay buffers: stores experience
- Algorithm interface → Replay buffers: samples experience
- Experience collection → Batch data: produces batches
- Training loop → Experiment logging: reports metrics
- Experiment logging → TensorBoard and W&B: writes logs
- RL evaluation → Experiment logging: reads experiment data