How genai-2025-project/yagang-yetanothergamengen- works
Project Type & Purpose
• This is an ML research pipeline for a single diffusion model that learns next-frame prediction across four games (Chess, Snake, Car Avoid, Game of Life).
• It comprises dataset generation, model finetuning/training, and inference.
File Structure
• Top level:
• Inference.ipynb – Jupyter notebook to run the trained model
• train.py – script to train the diffusion (UNet) model
• finetune_decoder.py – script to finetune the VAE decoder
• *_dataset.py – four scripts to generate raw datasets
• utils/ – helper modules for counting transitions, CSV conversion, visualization
• requirements.txt, LICENSE, README.md
• No web frontend or REST APIs; everything runs as Python scripts/notebooks.
Pull Key Details from README
• Model checkpoints and dataset hosted on Hugging Face.
• Uses Stable Diffusion 1.5V backbone: repurposes VAE encoder + UNet.
• Training uses accelerate launch train.py on multi-GPU.
• Inference runs in Jupyter, consumes GPU/ARM hardware.
Main Components & Their Relationships
a. Dataset Generation
• Four scripts (chess_dataset.py, snake_dataset.py, etc.)
• Output: raw image sequences + transition metadata stored in ~/dataset/.
• Utils support CSV conversion, transition counts, and visualization.
b. Model Preparation
• VAE Decoder Finetuning (finetune_decoder.py):
• Inputs: pre-trained Stable Diffusion VAE decoder + game images
• Outputs: finetuned VAE decoder weights
c. Model Training
• Training script (train.py):
• Inputs: finetuned VAE decoder, dataset, conditioning (previous frame + action)
• Framework: Hugging Face Diffusers + accelerate (multi-GPU)
• Outputs: UNet model checkpoints under ./training_diffusion/
d. Model Hosting & Storage
• Checkpoints & datasets pushed to Hugging Face Hub
• Local disk for intermediate artifacts (~/dataset/, ./training_diffusion/)
e. Inference Engine
• Jupyter Notebook (Inference.ipynb):
• Loads model checkpoints from local or HF
• Provides CLI-style interface to select game, sends actions, receives frames
• Requires ~8 GB VRAM or runs on ARM MacBooks
f. External Services
• Hugging Face Hub for hosting datasets & model checkpoints
• Python package index for dependencies (torch, diffusers, accelerate, etc.)
Architectural Patterns & Design Principles
• Pipeline pattern: clear separation of data generation → pretraining/finetuning → training → inference.
• Modularity: each game dataset script is interchangeable (single upstream pipeline).
• Reuse of foundation models (Stable Diffusion VAE + UNet) for multi-domain learning.