How ananya-ayasi/gpu-distributed works
Project type and purpose
• This is a hands-on PyTorch example repository for learning and demonstrating distributed (multi-GPU) model training and inference. It contains scripts, a config file, and a notebook to load data, define a GPT-2 style model, and orchestrate distributed training with PyTorch DDP.
File structure
• Top-level scripts:
• main.py – entry point that parses the YAML config and launches training
• trainer.py – encapsulates the training loop, batch distribution, and DDP setup
• model.py – defines the Transformer/GPT-2 style model architecture
• char_dataset.py – implements dataset creation and preprocessing from data/input.txt
• gpt2_train_cfg.yaml – training hyperparameters and DDP settings
• Distributed_Data_Parallel.ipynb – notebook demonstrating interactive usage
• Supporting files: data/input.txt as raw text corpus; requirements.txt for Python dependencies.