# guoyulong1/cleanrobotdqn architecture

> Architecture diagram of the GitHub repository guoyulong1/cleanrobotdqn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/guoyulong1/cleanrobotdqn
- Repository: https://github.com/guoyulong1/cleanrobotdqn

## Overview

##### Project Type and Purpose
- This is a focused reinforcement learning project to control a cleaning robot using a Deep Q-Network (DQN).
- The project involves reading a map (e.g., “my_office.txt” in the maps directory), configuring the environment, running a simulation (likely via main.py), and deploying the RL agent implemented in src/models (agent.py and dqn.py).
- The inclusion of a pre-trained model file (models/dqn_agent_map.pth) and reinforcement learning code in src/ indicates that the system’s heart is the RL agent and the environment simulation.

##### Analyzing the File Structure
- Top-level files:
- README.md tells us the project is “CleanRobotDQN.”
- main.py is likely the entry point where the simulation is initiated.
- requirements.txt specifies dependencies.
- Directory “maps” holds text-based representations of environments.
- Directory “models” holds a saved model (dqn_agent_map.pth), probably representing a trained agent.
- Directory “src” contains the bulk of the source code:
- config.py: Contains configuration settings and parameters.
- environment.py: Implements the simulation environment where the robot navigates.
- src/models: Contains agent.py and dqn.py, representing:
- The agent logic (decision making, action selection, etc.)
- The deep Q-Network (DQN) architecture and learning routines.
- src/utils/map_loader.py: Responsible for parsing and loading the map files.
- An empty “temp” directory might be used for intermediate data or logging, which is less central unless specified otherwise.

##### Core Components and Their Interactions
a. Main Application (main.py)
- Entry point that initializes the system.
- Loads configurations from src/config.py.
- Initiates the environment from src/environment.py.
- Loads a map via src/utils/map_loader.py which retrieves map definitions from the maps directory.
- Loads the pre-trained model from models/dqn_agent_map.pth if resuming or initializing the RL agent.
b. Environment Module (src/environment.py)
- Simulates the cleaning robot’s world, obstacles, and valid paths.
- Interacts with the RL agent by providing state information and receiving actions.
c. RL Agent and DQN (src/models/agent.py and src/models/dqn.py)
- The Agent manages the decision process, selecting actions based on the DQN.
- The DQN module defines the network architecture and learning algorithm.
- Agent interacts with the environment based on the reward feedback.
d. Configuration Module (src/config.py)
- Centralizes settings such as simulation parameters, learning rates, and other hyperparameters.
e. Utilities (src/utils/map_loader.py)
- Handles external data loading tasks (specifically map files) for the environment.
f. External Data and Models:
- The maps directory holds physical representations of the environment.
- The models directory holds a PyTorch model, indicating the use of a deep learning framework.

##### Directions
- Separation of concerns:
- Entry point (main.py) at the top, interfacing with all major modules.
- “Main Application (main.py)”
- “Configuration (config.py)”
- “Environment (environment.py)”
- “RL Agent (agent.py) and DQN (dqn.py)”
- “Map Loader Utility (map_loader.py)”
- “Data/External Resources (maps, models/dqn_agent_map.pth)”
- From main.py → loads configuration (config.py) and initializes Environment.
- Loading of the pre-trained model at initialization of the RL agent.
- Architectural patterns:
- The software follows a modular design typical in simulation/learning systems.
- Separation between the simulation (environment), configuration, model logic, and utilities.
- Notes on technologies:
- Python is the main language, and the existence of a .pth file suggests the use of PyTorch for the DQN.
- The structure supports iterative training/evaluation typical of reinforcement learning systems.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Main Application
    MA["Main Application (main.py)"]:::mainApp

    %% Core Modules
    CONF["Configuration (config.py)"]:::config
    ENV["Environment (environment.py)"]:::env
    ML["Map Loader Utility (map_loader.py)"]:::utils

    %% RL Agent Components Subgraph
    subgraph "RL Agent Components"
        RL["RL Agent (agent.py)"]:::rl
        DQN["Deep Q-Network (dqn.py)"]:::dqn
        RL --- DQN
    end

    %% External Data/Models
    MD["Map Data (my_office.txt)"]:::external
    PT["Pre-trained Model (dqn_agent_map.pth)"]:::external

    %% Connections from Main Application
    MA -->|"loadsConfig"| CONF
    MA -->|"initiatesEnv"| ENV
    MA -->|"fetchesMap"| ML

    %% Map Loader reads Map Data
    ML -->|"readsMap"| MD

    %% Pre-trained model connection to RL Agent
    PT -->|"loadsModel"| RL

    %% Bi-directional Interaction between Environment and RL Agent Components
    ENV <-->|"state/action"| RL

    %% Click Events
    click MA "https://github.com/guoyulong1/cleanrobotdqn/blob/main/main.py"
    click CONF "https://github.com/guoyulong1/cleanrobotdqn/blob/main/src/config.py"
    click ENV "https://github.com/guoyulong1/cleanrobotdqn/blob/main/src/environment.py"
    click RL "https://github.com/guoyulong1/cleanrobotdqn/blob/main/src/models/agent.py"
    click DQN "https://github.com/guoyulong1/cleanrobotdqn/blob/main/src/models/dqn.py"
    click ML "https://github.com/guoyulong1/cleanrobotdqn/blob/main/src/utils/map_loader.py"
    click MD "https://github.com/guoyulong1/cleanrobotdqn/blob/main/maps/my_office.txt"
    click PT "https://github.com/guoyulong1/cleanrobotdqn/blob/main/models/dqn_agent_map.pth"

    %% Styles
    classDef mainApp fill:#f9c74f,stroke:#333,stroke-width:2px;
    classDef config fill:#90be6d,stroke:#333,stroke-width:2px;
    classDef env fill:#f9844a,stroke:#333,stroke-width:2px;
    classDef rl fill:#577590,stroke:#333,stroke-width:2px;
    classDef dqn fill:#577590,stroke:#333,stroke-width:2px;
    classDef utils fill:#a9c0de,stroke:#333,stroke-width:2px;
    classDef external fill:#f94144,stroke:#333,stroke-width:2px;
```
