# gilesluo/dtr-bench architecture

> Architecture diagram of the GitHub repository gilesluo/dtr-bench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gilesluo/dtr-bench
- Repository: https://github.com/gilesluo/dtr-bench

## Overview

DTR-Bench is a reinforcement-learning benchmark for dynamic treatment regimes. Users configure an environment and policy, search hyperparameters, retrain/evaluate a selected policy, then visualize treatment effects. The graph separates search, learning and evaluation, and visualization, with shared policy and experiment utilities. The README documents continuous/discrete workflows and baseline evaluation; sampled excerpts show some wiring, while unsampled modules are represented only where documented. External environment implementations are outside this repository.

Main entry points: [`DTRBench/core/helper_fn.py`](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/helper_fn.py) (Algorithm dispatch), [`DTRBench/core/base_obj.py`](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/base_obj.py) (Training objective), [`DTRBench/run_rl/online_discrete_search.py`](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_discrete_search.py) (Hyperparameter search), [`DTRBench/run_rl/online_continuous_search.py`](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_continuous_search.py) (Continuous search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiments["Experiment workflows"]
  node_search["Hyperparameter search"]
  node_continuous_search["Continuous search"]
  node_retrain["Policy retraining"]
  node_continuous_retrain["Continuous retraining"]
  node_baseline_runner["Baseline evaluation"]
  node_study[("Optuna study")]
end

subgraph group_learning["RL learning"]
  node_hyperparams["Off-policy hyperparameters"]
  node_base_hparams["Common hyperparameters<br/>[base_hparams.py]"]
  node_objective["Training objective<br/>[base_obj.py]"]
  node_offpolicy["Off-policy algorithms<br/>[offpolicyRLObj.py]"]
  node_baseline_policy["Baseline policies<br/>[baseline_policy.py]"]
  node_networks["Policy networks<br/>[network.py]"]
end

subgraph group_shared["Shared utilities"]
  node_dispatch["Algorithm dispatch<br/>[helper_fn.py]"]
  node_policy_factory["Policy reconstruction<br/>[get_policy.py]"]
  node_data_utils["Training data utilities<br/>[data.py]"]
  node_tile_coding["Tile coding<br/>[tile_coding.py]"]
  node_result_utils["Result parsing<br/>[parse_result_fn.py]"]
  node_misc["Seed and trial helpers<br/>[misc.py]"]
  node_logger["Experiment logging<br/>[logger.py]"]
end

subgraph group_presentation["Visualization"]
  node_visual_entry["Visualization entry<br/>[visual.py]"]
  node_visualizer["Environment visualizer<br/>[env_vis_fn.py]"]
end

node_user(("Benchmark user"))
node_environment{{"DTRGym environment"}}

node_user -->|"configures search"| node_search
node_user -->|"configures search"| node_continuous_search
node_user -->|"starts training"| node_retrain
node_user -->|"starts training"| node_continuous_retrain
node_user -->|"requests baseline"| node_baseline_runner
node_search -->|"optimizes trials"| node_study
node_continuous_search -->|"optimizes trials"| node_study
node_search -->|"selects components"| node_dispatch
node_continuous_search -->|"selects components"| node_dispatch
node_dispatch -->|"returns class"| node_hyperparams
node_dispatch -->|"returns class"| node_offpolicy
node_hyperparams -->|"uses common parameters"| node_base_hparams
node_objective -->|"specializes objective"| node_offpolicy
node_offpolicy -->|"uses networks"| node_networks
node_objective -->|"trains and evaluates"| node_environment
node_retrain -->|"selects components"| node_dispatch
node_continuous_retrain -->|"selects components"| node_dispatch
node_retrain -.->|"runs training"| node_objective
node_continuous_retrain -.->|"runs training"| node_objective
node_baseline_runner -->|"evaluates policies"| node_baseline_policy
node_baseline_policy -->|"tests episodes"| node_environment
node_policy_factory -->|"builds policy"| node_networks
node_visual_entry -->|"calls visualizer"| node_visualizer
node_visualizer -->|"runs episodes"| node_environment
node_user -->|"requests plots"| node_visual_entry
node_objective -.->|"records metrics"| node_logger

click node_search "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_discrete_search.py"
click node_continuous_search "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_continuous_search.py"
click node_retrain "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_discrete_retrain.py"
click node_continuous_retrain "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_continuous_retrain.py"
click node_baseline_runner "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/baseline_discrete.py"
click node_dispatch "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/helper_fn.py"
click node_hyperparams "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/offpolicyRLHparams.py"
click node_base_hparams "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/base_hparams.py"
click node_objective "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/base_obj.py"
click node_offpolicy "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/offpolicyRLObj.py"
click node_baseline_policy "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/baseline_policy.py"
click node_policy_factory "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/get_policy.py"
click node_networks "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/network.py"
click node_data_utils "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/data.py"
click node_tile_coding "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/tile_coding.py"
click node_result_utils "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/parse_result_fn.py"
click node_misc "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/misc.py"
click node_logger "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/logger.py"
click node_visual_entry "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/visual_fn/visual.py"
click node_visualizer "https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/visual_fn/env_vis_fn.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_search,node_continuous_search,node_retrain,node_continuous_retrain,node_baseline_runner,node_study,node_user toneBlue
class node_hyperparams,node_base_hparams,node_objective,node_offpolicy,node_baseline_policy,node_networks toneAmber
class node_dispatch,node_policy_factory,node_data_utils,node_tile_coding,node_result_utils,node_misc,node_logger toneMint
class node_visual_entry,node_visualizer toneRose
class node_environment toneIndigo
```

## Components

### Experiment workflows

- [Hyperparameter search](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_discrete_search.py): `DTRBench/run_rl/online_discrete_search.py`
- [Continuous search](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_continuous_search.py): `DTRBench/run_rl/online_continuous_search.py`
- [Policy retraining](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_discrete_retrain.py): `DTRBench/run_rl/online_discrete_retrain.py`
- [Continuous retraining](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/online_continuous_retrain.py): `DTRBench/run_rl/online_continuous_retrain.py`
- [Baseline evaluation](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/run_rl/baseline_discrete.py): `DTRBench/run_rl/baseline_discrete.py`
- **Optuna study**

### RL learning

- [Off-policy hyperparameters](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/offpolicyRLHparams.py): `DTRBench/core/offpolicyRLHparams.py`
- [Common hyperparameters](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/base_hparams.py): `DTRBench/core/base_hparams.py`
- [Training objective](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/base_obj.py): `DTRBench/core/base_obj.py`
- [Off-policy algorithms](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/offpolicyRLObj.py): `DTRBench/core/offpolicyRLObj.py`
- [Baseline policies](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/baseline_policy.py): `DTRBench/core/baseline_policy.py`
- [Policy networks](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/network.py): `DTRBench/utils/network.py`

### Shared utilities

- [Algorithm dispatch](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/helper_fn.py): `DTRBench/core/helper_fn.py`
- [Policy reconstruction](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/get_policy.py): `DTRBench/utils/get_policy.py`
- [Training data utilities](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/data.py): `DTRBench/utils/data.py`
- [Tile coding](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/tile_coding.py): `DTRBench/utils/tile_coding.py`
- [Result parsing](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/parse_result_fn.py): `DTRBench/utils/parse_result_fn.py`
- [Seed and trial helpers](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/utils/misc.py): `DTRBench/utils/misc.py`
- [Experiment logging](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/core/logger.py): `DTRBench/core/logger.py`

### Visualization

- [Visualization entry](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/visual_fn/visual.py): `DTRBench/visual_fn/visual.py`
- [Environment visualizer](https://github.com/gilesluo/dtr-bench/blob/master/DTRBench/visual_fn/env_vis_fn.py): `DTRBench/visual_fn/env_vis_fn.py`

### Other components

- **Benchmark user**
- **DTRGym environment**

## Connections

- Benchmark user → Hyperparameter search: configures search (evidence: `README.md`)
- Benchmark user → Continuous search: configures search (evidence: `README.md`)
- Benchmark user → Policy retraining: starts training (evidence: `README.md`)
- Benchmark user → Continuous retraining: starts training (evidence: `README.md`)
- Benchmark user → Baseline evaluation: requests baseline (evidence: `README.md`)
- Hyperparameter search → Optuna study: optimizes trials (evidence: `DTRBench/run_rl/online_discrete_search.py`)
- Continuous search → Optuna study: optimizes trials (evidence: `DTRBench/run_rl/online_continuous_search.py`)
- Hyperparameter search → Algorithm dispatch: selects components (evidence: `DTRBench/run_rl/online_discrete_search.py`)
- Continuous search → Algorithm dispatch: selects components (evidence: `DTRBench/run_rl/online_continuous_search.py`)
- Algorithm dispatch → Off-policy hyperparameters: returns class (evidence: `DTRBench/core/helper_fn.py`)
- Algorithm dispatch → Off-policy algorithms: returns class (evidence: `DTRBench/core/helper_fn.py`)
- Off-policy hyperparameters → Common hyperparameters: uses common parameters (evidence: `DTRBench/core/offpolicyRLHparams.py`)
- Training objective → Off-policy algorithms: specializes objective (evidence: `DTRBench/core/offpolicyRLObj.py`)
- Off-policy algorithms → Policy networks: uses networks (evidence: `DTRBench/core/offpolicyRLObj.py`)
- Training objective → DTRGym environment: trains and evaluates (evidence: `README.md`)
- Policy retraining → Algorithm dispatch: selects components (evidence: `DTRBench/core/helper_fn.py`)
- Continuous retraining → Algorithm dispatch: selects components (evidence: `DTRBench/core/helper_fn.py`)
- Policy retraining → Training objective: runs training
- Continuous retraining → Training objective: runs training
- Baseline evaluation → Baseline policies: evaluates policies (evidence: `DTRBench/run_rl/baseline_discrete.py`)
- Baseline policies → DTRGym environment: tests episodes (evidence: `DTRBench/core/baseline_policy.py`)
- Policy reconstruction → Policy networks: builds policy (evidence: `DTRBench/utils/get_policy.py`)
- Visualization entry → Environment visualizer: calls visualizer (evidence: `DTRBench/visual_fn/visual.py`)
- Environment visualizer → DTRGym environment: runs episodes (evidence: `DTRBench/visual_fn/env_vis_fn.py`)
- Benchmark user → Visualization entry: requests plots (evidence: `README.md`)
- Training objective → Experiment logging: records metrics