# lollcat/rl-process-design architecture

> Architecture diagram of the GitHub repository lollcat/rl-process-design, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lollcat/rl-process-design
- Repository: https://github.com/lollcat/rl-process-design

## Overview

This repository explores reinforcement-learning-based chemical process synthesis through two documented case studies: choosing reactor sequences and designing distillation-column trains. The principal workflow is an agent selecting process operations and operating parameters, a simulator evaluating the resulting process and reward, and training/testing components refining or assessing the policy. The sampled continuous-action runner explicitly constructs the simulator and workers; the worker’s detailed environment/model calls are not fully shown, so those links are marked unverified. The README documents the reactor case study, while the tree and source index identify the distillation and additional reactor experiments.

Main entry points: [`Discrete and Continuous/main.py`](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/main.py) (Training runner), [`Discrete and Continuous/Workers/Worker_constrained_6.py`](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Workers/Worker_constrained_6.py) (RL worker), [`Discrete and Continuous/Env/Simulator_New.py`](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Simulator_New.py) (Distillation simulator), [`Discrete and Continuous/Env/Simulator_new_reward.py`](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Simulator_new_reward.py) (Alternative reward simulator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_reactor["Reactor studies"]
  node_sequence_case["Reactor sequencing<br/>[PFRorCSTR.py]"]
  node_cool_case["Reactor cooling<br/>[ReactorCool.py]"]
end

subgraph group_distillation["Distillation design"]
  node_simple_distill["Distillation agent"]
  node_sim_sparse["Distillation simulator<br/>[Simulator_New.py]"]
  node_sim_dense["Alternative reward simulator"]
end

subgraph group_learning["Learning runtime"]
  node_hybrid_runner["Training runner<br/>[main.py]"]
  node_worker["RL worker"]
  node_policy["Continuous policy<br/>[P_actor.py]"]
  node_q_network["Discrete Q network"]
  node_baseline_q["DQN variants<br/>[DQN.py]"]
  node_dueling_q["Dueling DQN<br/>[DQN_dueling.py]"]
  node_tester["Policy evaluator<br/>[tester.py]"]
  node_noise["Action exploration<br/>[OrnsteinNoise.py]"]
  node_memory["Experience memory<br/>[memory.py]"]
end

subgraph group_engineering["Process calculations"]
  node_sizing["Column sizing<br/>[column_sizing.py]"]
  node_capital["Capital costing<br/>[dc_capital_cost.py]"]
  node_annual["Operating costing<br/>[dc_annual_cost.py]"]
  node_sizing_sim["Sizing simulator"]
end

node_engineer(("Process designer"))

node_engineer -->|"specifies sequence"| node_sequence_case
node_engineer -.->|"starts training"| node_hybrid_runner
node_sequence_case -.->|"produces design"| node_engineer
node_hybrid_runner -->|"creates workers"| node_worker
node_hybrid_runner -->|"configures environment"| node_sim_sparse
node_hybrid_runner -->|"selects model"| node_policy
node_hybrid_runner -->|"selects model"| node_q_network
node_hybrid_runner -->|"selects model"| node_baseline_q
node_hybrid_runner -->|"selects model"| node_dueling_q
node_hybrid_runner -->|"imports evaluator"| node_tester
node_worker -.->|"steps environment"| node_sim_sparse
node_worker -.->|"updates policy"| node_policy
node_worker -.->|"updates Q network"| node_q_network
node_worker -->|"uses exploration"| node_noise
node_worker -->|"uses replay memory"| node_memory
node_tester -->|"evaluates policy"| node_sim_sparse
node_sim_sparse -->|"sizes columns"| node_sizing
node_sim_sparse -->|"calculates capital cost"| node_capital
node_sim_sparse -->|"calculates operating cost"| node_annual
node_sim_dense -->|"sizes columns"| node_sizing
node_sim_dense -->|"calculates capital cost"| node_capital
node_sim_dense -->|"calculates operating cost"| node_annual
node_sizing_sim -->|"uses sizing"| node_sizing
node_sizing_sim -->|"uses costing"| node_capital
node_sizing_sim -->|"uses costing"| node_annual
node_engineer -.->|"explores reactor design"| node_cool_case
node_engineer -.->|"explores distillation"| node_simple_distill

click node_sequence_case "https://github.com/lollcat/rl-process-design/blob/master/Discrete/PFR%20or%20CSTR/PFRorCSTR.py"
click node_cool_case "https://github.com/lollcat/rl-process-design/blob/master/Discrete/ReactOrCool/ReactorCool.py"
click node_simple_distill "https://github.com/lollcat/rl-process-design/blob/master/Discrete/Distillation/Distillation_simple1.py"
click node_hybrid_runner "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/main.py"
click node_worker "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Workers/Worker_constrained_6.py"
click node_policy "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/P_actor.py"
click node_q_network "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/DQN_dueling_new_structure.py"
click node_baseline_q "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/DQN.py"
click node_dueling_q "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/DQN_dueling.py"
click node_tester "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Utils/tester.py"
click node_noise "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Utils/OrnsteinNoise.py"
click node_memory "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Utils/memory.py"
click node_sim_sparse "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Simulator_New.py"
click node_sim_dense "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Simulator_new_reward.py"
click node_sizing "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/column_sizing.py"
click node_capital "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/dc_capital_cost.py"
click node_annual "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/dc_annual_cost.py"
click node_sizing_sim "https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/DistillationSimulator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sequence_case,node_cool_case toneBlue
class node_simple_distill,node_sim_sparse,node_sim_dense toneAmber
class node_hybrid_runner,node_worker,node_policy,node_q_network,node_baseline_q,node_dueling_q,node_tester,node_noise,node_memory toneMint
class node_sizing,node_capital,node_annual,node_sizing_sim toneRose
class node_engineer toneIndigo
```

## Components

### Reactor studies

- [Reactor sequencing](https://github.com/lollcat/rl-process-design/blob/master/Discrete/PFR%20or%20CSTR/PFRorCSTR.py): `Discrete/PFR or CSTR/PFRorCSTR.py`
- [Reactor cooling](https://github.com/lollcat/rl-process-design/blob/master/Discrete/ReactOrCool/ReactorCool.py): `Discrete/ReactOrCool/ReactorCool.py`

### Distillation design

- [Distillation agent](https://github.com/lollcat/rl-process-design/blob/master/Discrete/Distillation/Distillation_simple1.py): `Discrete/Distillation/Distillation_simple1.py`
- [Distillation simulator](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Simulator_New.py): `Discrete and Continuous/Env/Simulator_New.py`
- [Alternative reward simulator](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Simulator_new_reward.py): `Discrete and Continuous/Env/Simulator_new_reward.py`

### Learning runtime

- [Training runner](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/main.py): `Discrete and Continuous/main.py`
- [RL worker](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Workers/Worker_constrained_6.py): `Discrete and Continuous/Workers/Worker_constrained_6.py`
- [Continuous policy](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/P_actor.py): `Discrete and Continuous/Nets/P_actor.py`
- [Discrete Q network](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/DQN_dueling_new_structure.py): `Discrete and Continuous/Nets/DQN_dueling_new_structure.py`
- [DQN variants](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/DQN.py): `Discrete and Continuous/Nets/DQN.py`
- [Dueling DQN](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Nets/DQN_dueling.py): `Discrete and Continuous/Nets/DQN_dueling.py`
- [Policy evaluator](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Utils/tester.py): `Discrete and Continuous/Utils/tester.py`
- [Action exploration](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Utils/OrnsteinNoise.py): `Discrete and Continuous/Utils/OrnsteinNoise.py`
- [Experience memory](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Utils/memory.py): `Discrete and Continuous/Utils/memory.py`

### Process calculations

- [Column sizing](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/column_sizing.py): `Discrete and Continuous/Env/Sizing/column_sizing.py`
- [Capital costing](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/dc_capital_cost.py): `Discrete and Continuous/Env/Sizing/dc_capital_cost.py`
- [Operating costing](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/dc_annual_cost.py): `Discrete and Continuous/Env/Sizing/dc_annual_cost.py`
- [Sizing simulator](https://github.com/lollcat/rl-process-design/blob/master/Discrete%20and%20Continuous/Env/Sizing/DistillationSimulator.py): `Discrete and Continuous/Env/Sizing/DistillationSimulator.py`

### Other components

- **Process designer**

## Connections

- Process designer → Reactor sequencing: specifies sequence (evidence: `README.md`)
- Process designer → Training runner: starts training
- Reactor sequencing → Process designer: produces design
- Training runner → RL worker: creates workers (evidence: `Discrete and Continuous/main.py`)
- Training runner → Distillation simulator: configures environment (evidence: `Discrete and Continuous/main.py`)
- Training runner → Continuous policy: selects model (evidence: `Discrete and Continuous/main.py`)
- Training runner → Discrete Q network: selects model (evidence: `Discrete and Continuous/main.py`)
- Training runner → DQN variants: selects model (evidence: `Discrete and Continuous/main.py`)
- Training runner → Dueling DQN: selects model (evidence: `Discrete and Continuous/main.py`)
- Training runner → Policy evaluator: imports evaluator (evidence: `Discrete and Continuous/main.py`)
- RL worker → Distillation simulator: steps environment
- RL worker → Continuous policy: updates policy
- RL worker → Discrete Q network: updates Q network
- RL worker → Action exploration: uses exploration (evidence: `Discrete and Continuous/Workers/Worker_constrained_6.py`)
- RL worker → Experience memory: uses replay memory
- Policy evaluator → Distillation simulator: evaluates policy (evidence: `Discrete and Continuous/Utils/tester.py`)
- Distillation simulator → Column sizing: sizes columns (evidence: `Discrete and Continuous/Env/Simulator_New.py`)
- Distillation simulator → Capital costing: calculates capital cost (evidence: `Discrete and Continuous/Env/Simulator_New.py`)
- Distillation simulator → Operating costing: calculates operating cost (evidence: `Discrete and Continuous/Env/Simulator_New.py`)
- Alternative reward simulator → Column sizing: sizes columns (evidence: `Discrete and Continuous/Env/Simulator_new_reward.py`)
- Alternative reward simulator → Capital costing: calculates capital cost (evidence: `Discrete and Continuous/Env/Simulator_new_reward.py`)
- Alternative reward simulator → Operating costing: calculates operating cost (evidence: `Discrete and Continuous/Env/Simulator_new_reward.py`)
- Sizing simulator → Column sizing: uses sizing (evidence: `Discrete and Continuous/Env/Sizing/DistillationSimulator.py`)
- Sizing simulator → Capital costing: uses costing (evidence: `Discrete and Continuous/Env/Sizing/DistillationSimulator.py`)
- Sizing simulator → Operating costing: uses costing (evidence: `Discrete and Continuous/Env/Sizing/DistillationSimulator.py`)
- Process designer → Reactor cooling: explores reactor design
- Process designer → Distillation agent: explores distillation
