# matthewli-uw/autonomous-snake-agent architecture

> Architecture diagram of the GitHub repository matthewli-uw/autonomous-snake-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matthewli-uw/autonomous-snake-agent
- Repository: https://github.com/matthewli-uw/autonomous-snake-agent

## Overview

##### Project Overview
- The project is an implementation of a reinforcement learning system where a neural network-based agent learns to play the classic Snake game.

##### Main Components
- Game Engine (implemented in game.py)
- Simulates the Snake game environment.
- Handles game logic, state updates, food generation, collision detection, and scorekeeping.
- Agent (implemented in agent.py)
- Implements the reinforcement learning control loop.
- Manages experience collection, training iterations, and action selection.
- Neural Network Model (implemented in model.py)
- Contains the ViperNetwork which is a feedforward network with feature extraction layers.
- Uses a dueling architecture to derive separate value and advantage streams.
- Embeds core algorithms such as Double Q-Learning, PER, and gradient clipping.
- Plotting Module (implemented in plot.py)
- Visualizes metrics and training progress (e.g., scores or loss over episodes).
- Supporting Infrastructure:
- Configuration and environmental setup files (.gitignore, README.md).
- This project relies on external libraries (pygame for the game, PyTorch for neural networks, matplotlib for plotting, etc.).

##### Relationships and Data Flow
- The Agent interacts with the Game Engine:
- The game engine provides state information (like current snake position, food location, and danger detection).
- The agent receives a state representation (an 11-dimensional vector) from the game.
- The Agent feeds these states into the Neural Network Model:
- The model produces Q-values for the available actions (straight, right, left).
- The agent then uses these Q-values (with techniques such as Double Q-Learning) to decide on actions.
- Training Loop:
- The agent collects experiences and stores them in a prioritized replay buffer.
- Batch sampling, importance sampling, and loss calculation feed into the training routine.
- Periodically, the target network is synchronized with the online network.
- Data Visualization:
- Training stats and performance metrics are passed to the Plot module for rendering.

##### Architectural Patterns and Design Principles
- Clear separation of concerns:
- The game environment (game.py) is decoupled from the RL agent logic (agent.py) and learning modules (model.py).
- Reinforcement Learning Pipeline:
- The training loop encapsulates experience collection, learning updates, and target network synchronization.
- Modularity and Extensibility:
- Each component (game, model, plotting) is in its own file, making the system easy to extend or modify.
- Use of state-of-the-art RL techniques:

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Environment Layer"
        GE["Game Engine"]:::env
    end

    subgraph "Agent Layer"
        RA["RL Agent"]:::agent
        subgraph "Training Process"
            EXP["Experience Replay"]:::agentProc
            TL["Training Loop (DoubleQ-Learning, PER)"]:::agentProc
            TS["Target Network Sync"]:::agentProc
        end
    end

    subgraph "Model Layer"
        NN["Neural Network Model (ViperNetwork)"]:::model
    end

    subgraph "Visualization Layer"
        PM["Plot Module"]:::plot
    end

    GE -->|"providesState(11-dim)"| RA
    RA -->|"sendsStateToNN"| NN
    NN -->|"returnsQValues"| RA
    RA -->|"storesExperience"| EXP
    EXP -->|"samplesBatches"| TL
    TL -->|"updatesNetwork"| NN
    TL -->|"triggersSync"| TS
    RA -->|"sendsMetrics"| PM

    classDef env fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef agent fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef agentProc fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef model fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef plot fill:#e2e3e5,stroke:#383d41,stroke-width:2px;

    click GE "https://github.com/matthewli-uw/autonomous-snake-agent/blob/main/game.py"
    click RA "https://github.com/matthewli-uw/autonomous-snake-agent/blob/main/agent.py"
    click NN "https://github.com/matthewli-uw/autonomous-snake-agent/blob/main/model.py"
    click PM "https://github.com/matthewli-uw/autonomous-snake-agent/blob/main/plot.py"
```
