# nightfury217836/ai-learns-to-play-snake architecture

> Architecture diagram of the GitHub repository nightfury217836/ai-learns-to-play-snake, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nightfury217836/ai-learns-to-play-snake
- Repository: https://github.com/nightfury217836/ai-learns-to-play-snake

## Overview

##### Project type & purpose (from file tree + README)
This is a **single-process Python reinforcement learning project** that trains a **Deep Q-Network (DQN) agent** to play the classic **Snake** game “from scratch.” It’s not a distributed system or a web app; the “system architecture” here is primarily an **RL training pipeline** plus a **game environment** plus a **model persistence/inference path**.

Key indicators:
- `train.py` strongly implies a training entrypoint/orchestrator.
- `agent.py` implies the RL agent logic (policy, memory, action selection, training steps).
- `model.py` implies the neural network + training utilities (forward pass, optimizer hooks, etc.).
- `snake_game.py` implies the environment/simulator (state transitions, reward, terminal conditions).
- `plot_helper.py` implies training metrics visualization.
- `model/model.pth` implies saved weights/checkpoint.
- README explicitly says **Deep Q-Learning** and “AI learns Snake.”

##### How to read the repository structure into architectural components
Given the minimal tree, treat each file as a component boundary:

- **Entry points / Orchestration**
- `train.py`: coordinates episodes/steps, calls into Agent + Game, collects metrics, saves model.
- **Agent (RL logic)**
- `agent.py`: action selection (epsilon-greedy), experience replay buffer, training loop calls, target computation, exploration schedule.
- **Model (Deep Learning)**
- `model.py`: PyTorch model definition (Q-network), loss function, optimizer step, maybe a trainer class.
- `model/` directory: persistent artifacts
- `model.pth`: trained weights/checkpoint (output of training; input for inference).
- **Environment / Game**
- `snake_game.py`: game state, step/reset API (even if not formal Gym), reward, collision/terminal conditions, rendering.
- **Observability**
- `plot_helper.py`: plots scores, moving averages, loss curves, etc.

This project is best diagrammed as **dataflow** between these elements during training and during play/inference.

- **Control loop**: who calls whom each timestep
- **Data artifacts**: transitions stored, batches sampled, gradients applied
- **Model lifecycle**: initialization → training → checkpoint → load for inference
- **Feedback**: rewards/terminal signals drive learning

##### “Trainer / Main” (train.py)
Responsibilities:
- Initialize `SnakeGame`, `Agent`, `QNetwork`
- For each episode:
- reset env
- loop until done:
- get state
- ask agent for action
- step env
- push transition
- train step (short/long memory depending on implementation)
- log score, update epsilon schedule, save model periodically

##### “Environment” (snake_game.py)
Responsibilities:
- Maintain snake grid, direction, food placement
- Provide **state representation** to agent (often relative direction, danger flags, food direction, etc.)
- Implement `step(action) -> (next_state, reward, done, score)`
- Rendering (likely via pygame or similar)

##### “Agent” (agent.py)
Responsibilities:
- **Policy**: epsilon-greedy action selection using Q-network
- **Replay Memory**: store transitions `(state, action, reward, next_state, done)`
- **Training coordination**:
- sample mini-batch
- compute target Q values
- call model/trainer to backprop

##### “Q-Network / Trainer” (model.py)
Responsibilities:
- Neural net forward pass `Q(s, a)` for all actions
- Loss function (typically MSE/Huber between predicted Q and target Q)
- Optimizer step (Adam/SGD)
- Save/load weights

##### “Persistence” (model/model.pth)
Responsibilities:
- Durable artifact for trained weights
- Used by inference mode / later training sessions

##### “Metrics / Plotting” (plot_helper.py)
Responsibilities:
- Plot score per episode, running mean, possibly loss
- Acts as observability/feedback for training progress

##### train.py → snake_game.py
- Calls: `reset()`, `step(action)`, maybe `get_state()`, and render update.
##### train.py → agent.py
- Calls: `get_action(state)`, `remember(transition)`, `train_short_memory(...)`, `train_long_memory()`.
##### agent.py → model.py
- Calls: `predict(Q(s,*))`, `train_step(batch)` or equivalent.
##### snake_game.py → train.py / agent.py
- Returns: `next_state`, `reward`, `done`, `score`.
##### agent.py → agent.py (Replay Buffer internal loop)
- Store transition; sample mini-batch.
##### model.py → model/model.pth
- Saves checkpoint.
##### train.py → plot_helper.py
- Sends arrays of `scores`, `mean_scores` (and possibly `losses`) for plotting.

##### Inference/play loop (secondary dataflow)
Often implemented either in `agent.py` or a separate “play” mode inside `train.py`:

##### Technologies/frameworks to label
From typical DQN Snake implementations:
- **Python**
- **PyTorch** (very likely in `model.py`)
- **Pygame** or a similar renderer (very likely in `snake_game.py`)
- **Matplotlib** (likely in `plot_helper.py`)

- Timestep loop:
- state → action → step → (next_state, reward, done) → store transition → train step
- Episode boundary:
- long-memory training at end of episode (if used)
- plotting/logging
- checkpointing

Use labels like:
- “state vector”
- “action (discrete)”
- “reward, done, score”
- “transition tuple”
- “mini-batch sample”
- “Q-values”
- “loss/gradients”
- “weights checkpoint”

##### “Don’t overthink it”: separate into many components
Even though it’s single-process, break it down:
- Within **Agent**: Policy (epsilon-greedy), Replay Buffer, Batch Sampler
- Within **Model**: QNetwork, Loss, Optimizer, Save/Load
- Within **Environment**: State encoder, Reward function, Physics/collision, Renderer

- Top: `train.py` (orchestrator)
- Left: `snake_game.py` (environment + rendering)
- Center: `agent.py` (policy + replay buffer)
- Right: `model.py` (Q-network + optimizer)
- Bottom: `model/model.pth` (artifact) and `plot_helper.py` (metrics)

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================
  %% Level 1: Component Diagram
  %% =========================

  subgraph L1["Level 1: Static Components (Single-Process RL Project)"]
    direction TB

    README["README (Project Overview / Usage)"]:::doc

    subgraph ORCH["Orchestration"]
      direction TB
      Train["train.py (Trainer / Main Loop)"]:::module
    end

    subgraph ENVG["Environment (MDP Simulator)"]
      direction TB
      Env["snake_game.py (Snake Environment + Rendering)"]:::module
      EnvState["State Encoder (state vector schema)"]:::internal
      EnvReward["Reward + Terminal Conditions"]:::internal
      EnvRender["Renderer (Pygame)"]:::tool
      Env --> EnvState
      Env --> EnvReward
      Env --> EnvRender
    end

    subgraph AGENT["Agent (Decision + Learning Logic)"]
      direction TB
      Agent["agent.py (DQN Agent)"]:::module
      Policy["Policy (epsilon-greedy)"]:::internal
      Replay["Replay Memory (transition buffer)"]:::replay
      Sampler["Mini-batch Sampler"]:::internal
      Agent --> Policy
      Agent --> Replay
      Replay --> Sampler
    end

    subgraph MODEL["Function Approximator (Deep Learning)"]
      direction TB
      Model["model.py (Q-Network + Trainer)"]:::module
      QNet["Q-Network (PyTorch)"]:::tool
      LossOpt["Loss + Optimizer (training-only)"]:::trainonly
      TargetNet["Target Q-Network (optional)"]:::optional
      Model --> QNet
      Model --> LossOpt
      Model -.-> TargetNet
      TargetSync["Periodic Sync (optional)"]:::optional
      Model -.->|"copies_weights"| TargetSync -.-> TargetNet
    end

    subgraph OBS["Observability"]
      direction TB
      Plot["plot_helper.py (Metrics / Plotting - Matplotlib)"]:::module
      Charts["Score / Mean Score / Loss Curves"]:::tool
      Plot --> Charts
    end

    subgraph PERSIST["Persistence"]
      direction TB
      ModelDir["model/ (Artifacts Directory)"]:::dir
      Weights[("model.pth (Trained Weights / Checkpoint)")]:::artifact
      ModelDir --> Weights
    end
  end

  %% Relationships (static)
  README -->|"describes_usage"| Train

  Train -->|"reset()/step(action)"| Env
  Env -->|"next_state,reward,done,score"| Train

  Train -->|"get_action(state)"| Agent
  Train -->|"remember(transition)"| Agent
  Train -->|"train_short_memory(...) / train_long_memory()"| Agent

  Agent -->|"predict_Q(s,*)"| Model
  Agent -->|"train_step(mini_batch)"| Model
  Sampler -->|"mini-batch_sample"| Agent

  Model -.->|"save_checkpoint"| Weights
  Weights -.->|"load_weights_for_inference"| Model

  Train -.->|"scores,mean_scores,(losses)"| Plot

  %% Architectural seams (notes)
  Seam1["Contract Seam: state schema"]:::note
  Seam2["Contract Seam: action encoding (discrete)"]:::note
  Seam3["Contract Seam: reward + done conditions"]:::note
  EnvState -.-> Seam1
  Policy -.-> Seam2
  EnvReward -.-> Seam3

  %% =========================
  %% Level 2: Training / Inference Dataflow
  %% =========================

  subgraph L2["Level 2: Per-Timestep Dataflow (Training vs Inference)"]
    direction TB

    subgraph TRAINFLOW["Training Loop (per episode / per timestep)"]
      direction TB
      T0["Episode Start: reset()"]:::step
      T1["Get state vector"]:::step
      T2["Select action (epsilon-greedy)"]:::step
      T3["Env step(action)"]:::step
      T4["Receive reward,done,score + next_state"]:::step
      T5["Store transition tuple (s,a,r,s',done)"]:::step
      T6["Sample mini-batch (replay)"]:::step
      T7["Compute target Q + predicted Q"]:::step
      T8["Loss + backprop + optimizer step"]:::step
      T9["End of episode: train_long_memory(), log/plot, checkpoint"]:::step

      T0 --> T1 --> T2 --> T3 --> T4 --> T5 --> T6 --> T7 --> T8 --> T3
      T4 -->|"if_done"| T9
    end

    subgraph INFERFLOW["Inference / Play Loop (no training)"]
      direction TB
      I0["Load weights"]:::step
      I1["Get state vector"]:::step
      I2["Select action (epsilon≈0)"]:::step
      I3["Env step(action) + render"]:::step
      I4["Receive reward,done,score + next_state"]:::step
      I0 --> I1 --> I2 --> I3 --> I4 --> I1
    end
  end

  %% Wiring L2 to components (anchors)
  T0 -->|"calls_reset()"| Env
  T1 -->|"state_vector"| EnvState
  T2 -->|"action_selection"| Policy
  T3 -->|"action(discrete)"| Env
  T4 -->|"reward,done,score,next_state"| Train
  T5 -->|"transition_tuple"| Replay
  T6 -->|"mini_batch"| Sampler
  T7 -->|"Q_values/targets"| Model
  T8 -->|"loss/gradients"| LossOpt
  T9 -.->|"plot_metrics"| Plot
  T9 -.->|"save_checkpoint"| Weights

  I0 -->|"load_from"| Weights
  I2 -->|"uses_Q(s,*)"| Model
  I3 -->|"render"| EnvRender

  %% Click Events (from component_mapping)
  click Train "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/train.py"
  click Agent "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/agent.py"
  click Model "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/model.py"
  click Env "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/snake_game.py"
  click Plot "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/plot_helper.py"
  click Weights "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/model/model.pth"
  click ModelDir "https://github.com/nightfury217836/ai-learns-to-play-snake/tree/main/model/"
  click README "https://github.com/nightfury217836/ai-learns-to-play-snake/blob/main/README.md"

  %% Styles
  classDef module fill:#1f77b4,stroke:#0b3d91,stroke-width:2px,color:#ffffff;
  classDef internal fill:#87bfff,stroke:#0b3d91,stroke-width:1.5px,color:#0b1a33;
  classDef replay fill:#ff9f1c,stroke:#a85a00,stroke-width:2px,color:#2b1600;
  classDef tool fill:#2ecc71,stroke:#1e7e4a,stroke-width:2px,color:#0b2b18;
  classDef trainonly fill:#22c55e,stroke:#166534,stroke-width:2px,color:#052e16;
  classDef artifact fill:#9ca3af,stroke:#374151,stroke-width:2px,color:#111827;
  classDef dir fill:#cbd5e1,stroke:#475569,stroke-width:2px,color:#0f172a;
  classDef doc fill:#a855f7,stroke:#5b21b6,stroke-width:2px,color:#ffffff;
  classDef step fill:#0ea5e9,stroke:#075985,stroke-width:2px,color:#ffffff;
  classDef note fill:#fde68a,stroke:#92400e,stroke-width:1.5px,color:#451a03;
  classDef optional fill:#fca5a5,stroke:#7f1d1d,stroke-width:2px,color:#450a0a;
```
