# biswajitsadhu/raddqn architecture

> Architecture diagram of the GitHub repository biswajitsadhu/raddqn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/biswajitsadhu/raddqn
- Repository: https://github.com/biswajitsadhu/raddqn

## Overview

##### Project type and purpose
- RadDQN is an open-source, research-oriented reinforcement learning tool.  It implements a Deep Q-Learning (DQN) agent to compute time-efficient, radiation-minimizing paths through a gridworld populated with radioactive sources.  Its primary use case is to simulate scenarios with variable radiation fields and produce optimal traversal trajectories, comparing performance against a deterministic baseline (Dijkstra).

##### File structure
- Top-level files:
- GridBoard.py / Gridworld_v6static_3sources.py: gridworld environment definitions
- main.py: entry point that loads configs, builds components, and kicks off training
- trainer.py: orchestrates the RL training loop, policy updates, logging, and evaluation
- models/raddqn.py: definition of the neural network architecture and forward/update routines
- gridworld_config.py + configs/*.yaml: configuration loader and multiple experiment specifications
- run_script.sh: convenience wrapper to invoke main.py with appropriate flags
- info.log, LICENSE, README.md: documentation and metadata
- Patterns:
- A clear separation between configuration, environment, model, and training code
- Python modules for simulation vs. learning
- YAML-driven experiment parameters

##### README for additional insights
- Confirms architecture: environment state includes agent position, source positions and strengths
- Describes the custom reward function (radiation-aware) and novel exploration strategy
- Provides command-line interface to specify config, log directory, and random seed
- Includes high-level flowsheet showing interactions between agent, environment, reward module, and optimizer

a. Main components
1. Configuration Manager (gridworld_config.py + YAML files)
2. Environment Simulator (GridBoard / Gridworld_v6static_3sources)
3. Agent Core:
- Neural Network (models/raddqn.py)
- Exploration Strategy
- Replay Buffer (if implemented inside trainer)
4. Trainer / Orchestrator (trainer.py + main.py)
5. Logger / Metrics (writes info.log, console, optional tensorboard)
6. Runner Script (run_script.sh)
7. Baseline Comparator (Dijkstra module or external script)

b. Relationships & interactions
- The Runner Script invokes main.py, which:
1. Loads a YAML config via Configuration Manager
2. Instantiates the Environment Simulator
3. Builds the Agent Core (network + exploration)
4. Passes both to the Trainer
- During training, Trainer:
- Queries the Environment for next state & reward
- Feeds transitions to the Agent Core (for Q-value computation & updates)
- Updates network parameters
- Logs metrics after each episode
- After training, Trainer triggers an evaluation phase, optionally comparing against Dijkstra

c. Architectural patterns & design principles
- Modular separation: environment vs. model vs. orchestration
- Configuration-driven experiments (YAML)
- Clear dataflow: State → Network → Action → Environment → Reward → Update
- Decoupled logging and evaluation

d. Relevant technologies
- Python 3.x
- PyTorch or TensorFlow (in models/raddqn.py—check import)
- YAML for configs
- Shell script for reproducible runs
- Standard RL components (replay buffer, ε-greedy / custom exploration)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Runner Layer
    subgraph "Runner Layer"
        direction TB
        Runner["Runner Script\n(run_script.sh)"]:::compute
        Main["Main Entry Point\n(main.py)"]:::compute
    end

    Runner -->|"invokes"| Main

    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        ConfigMgr["Config Manager\n(gridworld_config.py)"]:::io
        ExperimentSpecs["Experiment Specs\n(configs/)"]:::io
    end

    Main -->|loads| ConfigMgr
    Main -->|loads| ExperimentSpecs

    %% Simulation Layer
    subgraph "Environment Simulator"
        direction TB
        EnvSim["Env Simulator\n(GridBoard.py)"]:::sim
        EnvAlt["Alt Env Simulator\n(Gridworld_v6static_3sources.py)"]:::sim
    end

    Main -->|instantiates| EnvSim
    Main -->|instantiates| EnvAlt

    %% Agent Core
    subgraph "Agent Core"
        direction TB
        AgentNet["Agent Neural Network\n(models/raddqn.py)"]:::compute
        Exploration["Exploration Strategy"]:::decision
    end

    Main -->|builds| AgentNet
    Main -->|builds| Exploration

    %% Orchestration
    subgraph "Trainer / Orchestrator"
        direction TB
        Trainer["Trainer / Orchestrator\n(trainer.py)"]:::compute
    end

    Main -->|passes to| Trainer

    %% Training Loop
    Trainer -->|requests state| EnvSim
    EnvSim -->|returns state| Trainer
    Trainer -->|queries reward| EnvSim
    EnvSim -->|returns reward| Trainer

    Trainer -->|computes action| AgentNet
    AgentNet -->|returns Q-values| Trainer
    Trainer -->|requests exploration decision| Exploration
    Exploration -->|returns action choice| Trainer

    Trainer -->|logs metrics| Logger
    Trainer -->|benchmarks| Baseline

    %% Output & Metadata
    subgraph "Output & Metadata"
        direction TB
        Logger["Logger / Metrics\n(info.log)"]:::storage
        Baseline["Baseline Comparator\n(Dijkstra)"]:::compute
        Documentation["Documentation & Metadata\n(README.md)"]:::io
        License["License\n(LICENSE)"]:::io
    end

    Trainer --> Logger
    Trainer --> Baseline

    %% Click Events
    click ConfigMgr "https://github.com/biswajitsadhu/raddqn/blob/main/gridworld_config.py"
    click ExperimentSpecs "https://github.com/biswajitsadhu/raddqn/tree/main/configs/"
    click EnvSim "https://github.com/biswajitsadhu/raddqn/blob/main/GridBoard.py"
    click EnvAlt "https://github.com/biswajitsadhu/raddqn/blob/main/Gridworld_v6static_3sources.py"
    click AgentNet "https://github.com/biswajitsadhu/raddqn/blob/main/models/raddqn.py"
    click Trainer "https://github.com/biswajitsadhu/raddqn/blob/main/trainer.py"
    click Main "https://github.com/biswajitsadhu/raddqn/blob/main/main.py"
    click Runner "https://github.com/biswajitsadhu/raddqn/blob/main/run_script.sh"
    click Logger "https://github.com/biswajitsadhu/raddqn/blob/main/info.log"
    click Documentation "https://github.com/biswajitsadhu/raddqn/blob/main/README.md"
    click License "https://github.com/biswajitsadhu/raddqn/tree/main/LICENSE"

    %% Styles
    classDef compute fill:#E3F2FD,stroke:#0D47A1,color:#0D47A1
    classDef io fill:#E8F5E9,stroke:#1B5E20,color:#1B5E20
    classDef sim fill:#FFF3E0,stroke:#E65100,color:#E65100
    classDef storage fill:#E1F5FE,stroke:#01579B,color:#01579B
    classDef decision fill:#FCE4EC,stroke:#880E4F,color:#880E4F
```
