How gilesluo/dtr-bench works
DTR-Bench is a reinforcement-learning benchmark for dynamic treatment regimes. Users configure an environment and policy, search hyperparameters, retrain/evaluate a selected policy, then visualize treatment effects. The graph separates search, learning and evaluation, and visualization, with shared policy and experiment utilities. The README documents continuous/discrete workflows and baseline evaluation; sampled excerpts show some wiring, while unsampled modules are represented only where documented. External environment implementations are outside this repository.
Main entry points: helper_fn.py Algorithm dispatch · base_obj.py Training objective · online_discrete_search.py Hyperparameter search · online_continuous_search.py Continuous search
Components 23
Experiment workflows
- Hyperparameter search
DTRBench/run_rl/online_discrete_search.py - Continuous search
DTRBench/run_rl/online_continuous_search.py - Policy retraining
DTRBench/run_rl/online_discrete_retrain.py - Continuous retraining
DTRBench/run_rl/online_continuous_retrain.py - Baseline evaluation
DTRBench/run_rl/baseline_discrete.py - Optuna study
RL learning
- Off-policy hyperparameters
DTRBench/core/offpolicyRLHparams.py - Common hyperparameters
DTRBench/core/base_hparams.py - Training objective
DTRBench/core/base_obj.py - Off-policy algorithms
DTRBench/core/offpolicyRLObj.py - Baseline policies
DTRBench/core/baseline_policy.py - Policy networks
DTRBench/utils/network.py
Shared utilities
- Algorithm dispatch
DTRBench/core/helper_fn.py - Policy reconstruction
DTRBench/utils/get_policy.py - Training data utilities
DTRBench/utils/data.py - Tile coding
DTRBench/utils/tile_coding.py - Result parsing
DTRBench/utils/parse_result_fn.py - Seed and trial helpers
DTRBench/utils/misc.py - Experiment logging
DTRBench/core/logger.py
Visualization
- Visualization entry
DTRBench/visual_fn/visual.py - Environment visualizer
DTRBench/visual_fn/env_vis_fn.py
Other components
- Benchmark user
- DTRGym environment
Connections 26
- Benchmark user → Hyperparameter search: configures search
README.md - Benchmark user → Continuous search: configures search
README.md - Benchmark user → Policy retraining: starts training
README.md - Benchmark user → Continuous retraining: starts training
README.md - Benchmark user → Baseline evaluation: requests baseline
README.md - Hyperparameter search → Optuna study: optimizes trials
DTRBench/run_rl/online_discrete_search.py - Continuous search → Optuna study: optimizes trials
DTRBench/run_rl/online_continuous_search.py - Hyperparameter search → Algorithm dispatch: selects components
DTRBench/run_rl/online_discrete_search.py - Continuous search → Algorithm dispatch: selects components
DTRBench/run_rl/online_continuous_search.py - Algorithm dispatch → Off-policy hyperparameters: returns class
DTRBench/core/helper_fn.py - Algorithm dispatch → Off-policy algorithms: returns class
DTRBench/core/helper_fn.py - Off-policy hyperparameters → Common hyperparameters: uses common parameters
DTRBench/core/offpolicyRLHparams.py - Training objective → Off-policy algorithms: specializes objective
DTRBench/core/offpolicyRLObj.py - Off-policy algorithms → Policy networks: uses networks
DTRBench/core/offpolicyRLObj.py - Training objective → DTRGym environment: trains and evaluates
README.md - Policy retraining → Algorithm dispatch: selects components
DTRBench/core/helper_fn.py - Continuous retraining → Algorithm dispatch: selects components
DTRBench/core/helper_fn.py - Policy retraining → Training objective: runs training
- Continuous retraining → Training objective: runs training
- Baseline evaluation → Baseline policies: evaluates policies
DTRBench/run_rl/baseline_discrete.py - Baseline policies → DTRGym environment: tests episodes
DTRBench/core/baseline_policy.py - Policy reconstruction → Policy networks: builds policy
DTRBench/utils/get_policy.py - Visualization entry → Environment visualizer: calls visualizer
DTRBench/visual_fn/visual.py - Environment visualizer → DTRGym environment: runs episodes
DTRBench/visual_fn/env_vis_fn.py - Benchmark user → Visualization entry: requests plots
README.md - Training objective → Experiment logging: records metrics