How biswajitsadhu/raddqn works
Project type and purpose
• RadDQN is an open-source, research-oriented reinforcement learning tool. It implements a Deep Q-Learning (DQN) agent to compute time-efficient, radiation-minimizing paths through a gridworld populated with radioactive sources. Its primary use case is to simulate scenarios with variable radiation fields and produce optimal traversal trajectories, comparing performance against a deterministic baseline (Dijkstra).
File structure
• Top-level files:
• GridBoard.py / Gridworld_v6static_3sources.py: gridworld environment definitions
• main.py: entry point that loads configs, builds components, and kicks off training
• trainer.py: orchestrates the RL training loop, policy updates, logging, and evaluation
• models/raddqn.py: definition of the neural network architecture and forward/update routines
• gridworld_config.py + configs/*.yaml: configuration loader and multiple experiment specifications
• run_script.sh: convenience wrapper to invoke main.py with appropriate flags
• info.log, LICENSE, README.md: documentation and metadata
• Patterns:
• A clear separation between configuration, environment, model, and training code
• Python modules for simulation vs. learning
• YAML-driven experiment parameters
README for additional insights
• Confirms architecture: environment state includes agent position, source positions and strengths
• Describes the custom reward function (radiation-aware) and novel exploration strategy
• Provides command-line interface to specify config, log directory, and random seed
• Includes high-level flowsheet showing interactions between agent, environment, reward module, and optimizer
a. Main components
1. Configuration Manager (gridworld_config.py + YAML files)
2. Environment Simulator (GridBoard / Gridworld_v6static_3sources)
3. Agent Core:
• Neural Network (models/raddqn.py)
• Exploration Strategy
• Replay Buffer (if implemented inside trainer)
4. Trainer / Orchestrator (trainer.py + main.py)
5. Logger / Metrics (writes info.log, console, optional tensorboard)
6. Runner Script (run_script.sh)
7. Baseline Comparator (Dijkstra module or external script)
b. Relationships & interactions
• The Runner Script invokes main.py, which:
1. Loads a YAML config via Configuration Manager
2. Instantiates the Environment Simulator
3. Builds the Agent Core (network + exploration)
4. Passes both to the Trainer
• During training, Trainer:
• Queries the Environment for next state & reward
• Feeds transitions to the Agent Core (for Q-value computation & updates)
• Updates network parameters
• Logs metrics after each episode
• After training, Trainer triggers an evaluation phase, optionally comparing against Dijkstra
c. Architectural patterns & design principles
• Modular separation: environment vs. model vs. orchestration
• Configuration-driven experiments (YAML)
• Clear dataflow: State → Network → Action → Environment → Reward → Update
• Decoupled logging and evaluation
d. Relevant technologies
• Python 3.x
• PyTorch or TensorFlow (in models/raddqn.py—check import)
• YAML for configs
• Shell script for reproducible runs
• Standard RL components (replay buffer, ε-greedy / custom exploration)