# hatoym/coaching-dqn architecture

> Architecture diagram of the GitHub repository hatoym/coaching-dqn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hatoym/coaching-dqn
- Repository: https://github.com/hatoym/coaching-dqn

## Overview

##### Main Components
- Entry Point & Controller: The main.py file serves as the entry point. It likely initializes the agents, loads the trained networks, and starts the simulation or training/evaluation loop.
- Agent Module: Inside the “agent” directory, you have different submodules:
– adviser.py: Contains logic for the pre-trained adviser (possibly rule-based or DQN-based decision-making).
– player.py: Contains logic for the untrained, learning player.
– common.py and make_adviser.py: Utilities that handle shared operations or the creation/configuration of the advising agent.
- Trained Networks Repository: The “trained_adviser” directory holds saved network models for different versions (e.g., Breakout-v0, BreakoutNoFrameskip-v4) that are used by the adviser. Files such as data, index, meta, checkpoints, and events indicate the TensorFlow model checkpoints and logging.
- External Dependencies: The project uses an external DQN implementation. This should be indicated as an external library or service that offers deep Q-network (DQN) functionalities.

##### Architectural Relationships and Data Flow
- The main.py acts as the orchestrator—it initializes the agent components, loads the pretrained models from the “trained_adviser” folder, and then proceeds with a game simulation or training process.
- Also, show that external libraries (the DQN repository) support the core learning functionality inside one or both agents.

##### Architectural Patterns and Design Principles
- Modular Design: The separation between adviser and player modules follows a modular approach, which allows independent development and testing.
- Model & Policy Separation: The trained adherents (network files) suggest that the system separates model training from inference. Include a component that represents the “trained network” model loaded into the adviser.
- Reuse of External Code: The system relies on a DQN implementation that needs to be represented as an external module.

##### Elements
– Initialization and configuration flow from main.py to the agent modules.
– Data flow from the game environment to the player and adviser.
– Loading the pretrained network from the “trained_adviser” directory into the adviser component.
- Optionally, annotate details about important configuration files (like requirements.txt) that ensure consistent dependency management.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Main Controller (main.py)"]:::main
    subgraph "Agent Modules"
        subgraph "Adviser Module"
            ADV["Adviser Core\\n(agent/adviser.py)"]:::adviser
            ADV_CONFIG["Adviser Config\\n(agent/make_adviser.py)"]:::adviser
            ADV_UTILS["Adviser Utils\\n(agent/common.py)"]:::adviser
        end
        PLY["Player Agent\\n(agent/player.py)"]:::player
    end
    ENV["Game Environment"]:::env
    TM["Trained Model Storage\\n(trained_adviser"]:::storage
    EXT["External DQN Library\\n(requirements.txt)"]:::external

    %% Connections from Main Controller to Agents
    A -->|"init_agents"| ADV
    A -->|"init_agents"| PLY

    %% Connections from Game Environment to Agents
    ENV -->|"provides_state"| ADV
    ENV -->|"provides_state"| PLY

    %% Adviser internal support connections
    ADV_CONFIG -->|"configures"| ADV
    ADV_UTILS -->|"supports"| ADV

    %% Adviser loads pretrained models and uses external DQN
    ADV -->|"loads_model"| TM
    ADV -->|"utilizes_DQN"| EXT

    %% Styles
    classDef main fill:#f9c74f,stroke:#333,stroke-width:2px;
    classDef adviser fill:#90be6d,stroke:#333,stroke-width:2px;
    classDef player fill:#f94144,stroke:#333,stroke-width:2px;
    classDef storage fill:#577590,stroke:#333,stroke-width:2px,stroke-dasharray: 5,5;
    classDef external fill:#277da1,stroke:#333,stroke-width:2px,stroke-dasharray: 2,2;
    classDef env fill:#e9ecef,stroke:#333,stroke-width:2px;

    %% Click Events
    click A "https://github.com/hatoym/coaching-dqn/blob/atari/instruct_and_nointent/main.py"
    click ADV "https://github.com/hatoym/coaching-dqn/blob/atari/instruct_and_nointent/agent/adviser.py"
    click ADV_CONFIG "https://github.com/hatoym/coaching-dqn/blob/atari/instruct_and_nointent/agent/make_adviser.py"
    click ADV_UTILS "https://github.com/hatoym/coaching-dqn/blob/atari/instruct_and_nointent/agent/common.py"
    click PLY "https://github.com/hatoym/coaching-dqn/blob/atari/instruct_and_nointent/agent/player.py"
    click TM "https://github.com/hatoym/coaching-dqn/tree/atari/instruct_and_nointent/trained_adviser"
    click EXT "https://github.com/hatoym/coaching-dqn/blob/atari/instruct_and_nointent/requirements.txt"
```
