# anirudhvijayabalaji/reinforcement-learning architecture

> Architecture diagram of the GitHub repository anirudhvijayabalaji/reinforcement-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anirudhvijayabalaji/reinforcement-learning
- Repository: https://github.com/anirudhvijayabalaji/reinforcement-learning

## Overview

MolRL generates molecules in protein binding pockets through interactive or training workflows: retrieve and voxelize a target, propose atom placements, validate and score the assembled molecule, then inspect or compare results. The graph includes the documented web and desktop interfaces, agent comparison, interaction profiling, and explanation features. Edges are limited to source-index references, shown excerpts, and README statements; several documented subsystems were not sampled, so their internal connections are not inferred.

Main entry points: [`app.py`](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/app.py) (MolRL Studio), [`environment.py`](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/environment.py) (Placement environment), [`main.py`](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/main.py) (Training orchestration), [`visualizer.py`](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/visualizer.py) (Tkinter dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_web_app["MolRL Studio<br/>[app.py]"]
  node_browser["Browser visualization<br/>[app.js]"]
  node_desktop["Tkinter dashboard<br/>[visualizer.py]"]
  node_visualization["Molecule visualization<br/>[visualization.py]"]
end

subgraph group_generation["Molecule generation"]
  node_environment["Placement environment<br/>[environment.py]"]
end

subgraph group_learning["Learning and agents"]
  node_training["Training orchestration<br/>[main.py]"]
  node_policy["3D actor-critic<br/>[agent.py]"]
  node_attention["3D attention blocks<br/>[cbam_3d.py]"]
  node_ppo["PPO optimization<br/>[ppo.py]"]
  node_agent_zoo["Agent zoo<br/>[agents_zoo.py]"]
  node_benchmark["Agent benchmark<br/>[benchmark.py]"]
end

subgraph group_chemistry["Chemistry and scoring"]
  node_molecule["Molecule construction and validation<br/>[physics.py]"]
  node_reward["Target reward oracles<br/>[reward.py]"]
  node_formula["Formula explanations"]
end

subgraph group_protein["Protein and analysis"]
  node_target_config["Target configuration<br/>[target_config.json]"]
  node_pdb["Structure retrieval and voxelization<br/>[pdb_parser.py]"]
  node_interactions["Pocket interaction profiling<br/>[interactions.py]"]
end

node_web_user(("Web user"))
node_desktop_user(("Desktop user"))
node_rcsb["RCSB and AlphaFold"]

node_web_user -->|"uses"| node_web_app
node_web_app -->|"serves visualization"| node_browser
node_desktop_user -->|"uses"| node_desktop
node_training -->|"steps through"| node_environment
node_training -->|"optimizes with"| node_ppo
node_training -->|"trains"| node_policy
node_training -->|"generates outputs"| node_visualization
node_environment -->|"loads pocket"| node_pdb
node_environment -->|"constructs molecule"| node_molecule
node_environment -->|"scores molecule"| node_reward
node_target_config -->|"selects target"| node_environment
node_pdb -->|"retrieves structures"| node_rcsb
node_policy -->|"applies"| node_attention
node_ppo -->|"updates"| node_policy
node_web_app -->|"steps through"| node_environment
node_web_app -->|"selects agent"| node_agent_zoo
node_web_app -->|"starts benchmark"| node_benchmark
node_web_app -->|"explains assembly"| node_formula
node_web_app -->|"profiles contacts"| node_interactions
node_web_app -->|"shows scores"| node_reward
node_desktop -->|"steps through"| node_environment
node_desktop -->|"loads target"| node_pdb
node_desktop -->|"runs training"| node_ppo
node_interactions -->|"uses chemistry"| node_molecule
node_benchmark -->|"compares agents"| node_agent_zoo
node_browser -.->|"renders molecules"| node_visualization

click node_web_app "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/app.py"
click node_browser "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/static/js/app.js"
click node_desktop "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/visualizer.py"
click node_training "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/main.py"
click node_environment "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/environment.py"
click node_target_config "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/target_config.json"
click node_pdb "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/pdb_parser.py"
click node_policy "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/agent.py"
click node_attention "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/cbam_3d.py"
click node_ppo "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/ppo.py"
click node_agent_zoo "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/agents_zoo.py"
click node_benchmark "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/benchmark.py"
click node_molecule "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/physics.py"
click node_reward "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/reward.py"
click node_formula "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/formula_explainer.py"
click node_interactions "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/interactions.py"
click node_visualization "https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/visualization.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web_app,node_browser,node_desktop,node_visualization,node_web_user,node_desktop_user toneBlue
class node_environment toneAmber
class node_training,node_policy,node_attention,node_ppo,node_agent_zoo,node_benchmark toneMint
class node_molecule,node_reward,node_formula toneRose
class node_target_config,node_pdb,node_interactions,node_rcsb toneIndigo
```

## Components

### User interfaces

- [MolRL Studio](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/app.py): `app.py`
- [Browser visualization](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/static/js/app.js): `static/js/app.js`
- [Tkinter dashboard](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/visualizer.py): `visualizer.py`
- [Molecule visualization](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/visualization.py): `visualization.py`

### Molecule generation

- [Placement environment](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/environment.py): `environment.py`

### Learning and agents

- [Training orchestration](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/main.py): `main.py`
- [3D actor-critic](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/agent.py): `agent.py`
- [3D attention blocks](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/cbam_3d.py): `cbam_3d.py`
- [PPO optimization](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/ppo.py): `ppo.py`
- [Agent zoo](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/agents_zoo.py): `agents_zoo.py`
- [Agent benchmark](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/benchmark.py): `benchmark.py`

### Chemistry and scoring

- [Molecule construction and validation](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/physics.py): `physics.py`
- [Target reward oracles](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/reward.py): `reward.py`
- [Formula explanations](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/formula_explainer.py): `formula_explainer.py`

### Protein and analysis

- [Target configuration](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/target_config.json): `target_config.json`
- [Structure retrieval and voxelization](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/pdb_parser.py): `pdb_parser.py`
- [Pocket interaction profiling](https://github.com/anirudhvijayabalaji/reinforcement-learning/blob/main/interactions.py): `interactions.py`

### Other components

- **Web user**
- **Desktop user**
- **RCSB and AlphaFold**

## Connections

- Web user → MolRL Studio: uses (evidence: `README.md`)
- MolRL Studio → Browser visualization: serves visualization (evidence: `README.md`)
- Desktop user → Tkinter dashboard: uses (evidence: `README.md`)
- Training orchestration → Placement environment: steps through (evidence: `main.py`)
- Training orchestration → PPO optimization: optimizes with (evidence: `main.py`)
- Training orchestration → 3D actor-critic: trains (evidence: `main.py`)
- Training orchestration → Molecule visualization: generates outputs (evidence: `main.py`)
- Placement environment → Structure retrieval and voxelization: loads pocket (evidence: `environment.py`)
- Placement environment → Molecule construction and validation: constructs molecule (evidence: `environment.py`)
- Placement environment → Target reward oracles: scores molecule (evidence: `environment.py`)
- Target configuration → Placement environment: selects target (evidence: `environment.py`)
- Structure retrieval and voxelization → RCSB and AlphaFold: retrieves structures (evidence: `pdb_parser.py`)
- 3D actor-critic → 3D attention blocks: applies (evidence: `agent.py`)
- PPO optimization → 3D actor-critic: updates (evidence: `main.py`)
- MolRL Studio → Placement environment: steps through (evidence: `app.py`)
- MolRL Studio → Agent zoo: selects agent (evidence: `app.py`)
- MolRL Studio → Agent benchmark: starts benchmark (evidence: `app.py`)
- MolRL Studio → Formula explanations: explains assembly (evidence: `README.md`)
- MolRL Studio → Pocket interaction profiling: profiles contacts (evidence: `README.md`)
- MolRL Studio → Target reward oracles: shows scores (evidence: `app.py`)
- Tkinter dashboard → Placement environment: steps through (evidence: `visualizer.py`)
- Tkinter dashboard → Structure retrieval and voxelization: loads target (evidence: `visualizer.py`)
- Tkinter dashboard → PPO optimization: runs training (evidence: `visualizer.py`)
- Pocket interaction profiling → Molecule construction and validation: uses chemistry (evidence: `interactions.py`)
- Agent benchmark → Agent zoo: compares agents (evidence: `README.md`)
- Browser visualization → Molecule visualization: renders molecules (evidence: `README.md`)
