How bhavya0304/snakeai works
Project type and purpose
• This is a research/demo project that implements Deep Q-Learning to train an AI agent on the Snake game.
• Key goal: demonstrate RL training loop, environment interaction, and model persistence/visualization.
File structure
Top-level:
• AI.py, SnakeAI.py, network.py, app.py – standalone scripts for experiments or launching games.
• requirements.txt – lists Python dependencies (e.g., pygame, torch).
• research/Reinforcement-Learning – background notes or references.
Core module: SnakeAI_Final/
• agent.py – DQN agent logic (action selection, ε-greedy policy, training loop).
• game.py – custom Snake environment built on pygame (state, reward, reset, step).
• helper.py – utility functions (e.g., state preprocessing, reward shaping).
• model.py – neural network definition (PyTorch module) and model persistence (load/save).
• model/model.pth – checkpoint of trained network.
Extract architecture insights from the README
• Components described: Environment, State Representation, Action Space, Reward Function, DQN, Experience Replay, Target Network.
• Training flow steps and dataflow between agent and environment.
Main components
a. Snake Game Environment (game.py + Pygame)
b. Replay Buffer (experience storage – implemented in agent.py/helper)
c. Deep Q-Network (model.py + network.py)
d. Target Network (a copy of the DQN, updated periodically)
e. Training Loop (agent.py orchestrates episodes & optimization)
f. Model Persistence (saving/loading model.pth)
g. Visualization / Demo App (app.py or SnakeAI.py for real-time display)
Relationships and dataflow
1. Training Loop triggers:
– “Reset Environment” → game.py
– “Observe State” ← returns state
– “Select Action (ε-greedy)” → agent.py
– “Step Environment(Action)” → game.py → returns next state, reward, done flag
– “Store Experience” → Replay Buffer
– “Sample Mini-batch” → Replay Buffer → feed to DQN
– “Compute Target Q-values” (using Target Network)
– “Optimize DQN” (backprop in model.py)
2. “Update Target Network” periodically (agent.py)
3. “Save Model Checkpoint” → disk (model/model.pth)
4. “Visualization” (app.py) reads model and runs live demo
Architectural patterns & principles
• Separation of Concerns:
• Environment vs. Agent vs. Network definition
• Replay Buffer decouples data collection from learning
• Modular Design: each Python module has a clear responsibility
• Pipeline Pattern: data flows in a loop (state → action → reward → learning)
Essential elements
• Component labels matching project files.
• Arrowheads showing flow direction.
• A note on the ε-greedy policy and target network update frequency.