# analoganddigital/dqn_play_sekiro architecture

> Architecture diagram of the GitHub repository analoganddigital/dqn_play_sekiro, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/analoganddigital/dqn_play_sekiro
- Repository: https://github.com/analoganddigital/dqn_play_sekiro

## Overview

Based on both the file tree and README, this project is a specialized reinforcement learning (RL) system that uses Deep Q Networks (DQN) to let an agent play the game Sekiro. The system’s overall purpose is to use real-time game screen data to provide feedback (reward signals) while the DQN agent learns to play the game by interacting with it using simulated keystrokes.

##### Main Components
- Data Acquisition Module: Represent the “grabscreen.py” component which continuously captures the game’s screen. This is the entry point for visual data.
- Reward and State Processing: Include “find_blood_location.py” since this module processes the captured images to extract health bar positions and compute rewards that guide the learning process.
- Input Simulation Module: Show the “directkeys.py” and “getkeys.py” components that convert the DQN decisions into keystroke events to interact with the game.
- Game Management/Restart Module: “restart.py” is used to quickly restart the game (leveraging the game modifier) and ensure continuous training episodes.
- Legacy or Alternative Implementation: Optionally note “DQN_tensorflow_gpu.py” as an alternative implementation or historical reference to past supervised learning approaches.

##### Relationships and Data Flow
– One indicating the output decision (action) directed toward the Input Simulation module (“directkeys.py” and “getkeys.py”) which then provides simulated keystroke commands to the game.
– Another feedback loop that records the interaction outcomes (game state after an action) and updates the learning process.
- Finally, include a component for Game Restart (“restart.py”) that interfaces with the game environment to ensure minimal downtime, connecting back to the Screen Capture module for a new training episode.

##### Architectural Patterns and Design Principles
- Reinforcement Learning Loop: Illustrate the cyclical nature of the RL feedback loop (state capture → action decision → game simulation → reward determination → learning update).
- Integration with External Modifiers: Note that the system depends on a game modifier (specified in the README) to handle the game restart efficiently. Highlight this as an external dependency.

##### Technologies and Frameworks
- That the DQN training leverages GPU capabilities (as indicated by the “_gpu” suffix in most scripts) and it is built on Python with TensorFlow (or an alternative neural network framework, given the mention of AlexNet in the README).
- Although not explicitly shown in the file tree, include a note on the use of reinforcement learning principles, along with the AlexNet architecture for the visual component, which is part of the proposed system architecture.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External System
    GE["Game Environment"]:::external

    %% Data Acquisition and Processing
    SC["Screen Capture (grabscreen.py)"]:::data
    RP["Reward Extraction (find_blood_location.py)"]:::data

    %% DQN Agent Subgraph
    subgraph "DQN Agent"
        DQN_T["DQN Agent Training (DQN_sekiro_training_gpu.py)"]:::core
        DQN_Test["DQN Agent Testing (DQN_sekiro_testing_gpu.py)"]:::core
        Legacy["Legacy DQN (DQN_tensorflow_gpu.py)"]:::legacy
        DO["Decision Output"]:::core
    end

    %% Input Simulation Subgraph
    subgraph "Input Simulation Module"
        DK["Direct Keys (directkeys.py)"]:::action
        GK["Get Keys (getkeys.py)"]:::action
    end

    %% Game Management/Restart
    GR["Game Restart (restart.py)"]:::management

    %% Flow Connections
    GE -->|"capture"| SC
    SC -->|"image"| RP
    RP -->|"state"| DQN_T
    RP -->|"state"| DQN_Test
    RP -->|"state"| Legacy

    DQN_T -->|"decision"| DO
    DQN_Test -->|"decision"| DO
    Legacy -->|"decision"| DO

    DO -->|"action"| DK
    DO -->|"action"| GK

    DK -->|"simulate"| GE
    GK -->|"simulate"| GE

    %% Integration with Game Restart Module
    DQN_T ---|"episode-reset"| GR
    DQN_Test ---|"episode-reset"| GR
    DO ---|"feedback"| GR
    GR -->|"restart"| SC

    %% Click Events
    click SC "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/grabscreen.py"
    click RP "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/find_blood_location.py"
    click DQN_T "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/DQN_sekiro_training_gpu.py"
    click DQN_Test "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/DQN_sekiro_testing_gpu.py"
    click Legacy "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/DQN_tensorflow_gpu.py"
    click DK "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/directkeys.py"
    click GK "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/getkeys.py"
    click GR "https://github.com/analoganddigital/dqn_play_sekiro/blob/main/restart.py"

    %% Style Definitions
    classDef external fill:#ff9999,stroke:#333,stroke-width:2px;
    classDef data fill:#99ccff,stroke:#333,stroke-width:2px;
    classDef core fill:#99ff99,stroke:#333,stroke-width:2px;
    classDef action fill:#ffcc66,stroke:#333,stroke-width:2px;
    classDef management fill:#cccccc,stroke:#333,stroke-width:2px;
    classDef legacy fill:#ffcccc,stroke:#333,stroke-width:2px;
```
