How bes-dev/mobilestylegan.pytorch works
Project type and purpose
• This is a PyTorch-based machine-learning framework (“MobileStyleGAN”) for training, evaluating, converting, and deploying a lightweight StyleGAN2 variant (MobileStyleGAN) for high-fidelity image synthesis on edge devices.
• Key use cases:
• Train a student GAN via distillation from a teacher StyleGAN2
• Convert existing StyleGAN2 checkpoints into the MobileStyleGAN format
• Generate images, evaluate FID, run demos/comparisons
• Export trained models to ONNX/CoreML and deploy with OpenVINO
File structure
• Top-level scripts (entry points):
• train.py (training & export)
• generate.py (batch image generation)
• demo.py / compare.py (visual demos & side-by-side comparisons)
• evaluate_fid.py (FID score evaluation)
• convert_rosinality_ckpt.py (import rosinality StyleGAN2 ckpts)
• configs/ – JSON configuration for model architectures, training hyperparameters, dataset paths, export flags
• core/ – core framework modules:
• dataset.py (data loader & augmentation)
• distiller.py (teacher → student knowledge distillation logic)
• loss/ (GAN, distillation, perceptual, diffaug losses)
• models/ (network definitions)
– mapping_network, mobile_synthesis_network (student), discriminator, inception_v3 (aux for perceptual loss)
– modules/ (building blocks: modulated_conv2d, up/downsampling, wavelets, custom CUDA kernels)
• requirements.txt – external dependencies
• res/ – sample output images
• LICENSE-NVIDIA – CUDA kernel license
README for additional insights
• Describes each script’s usage, required GPUs, external library integrations (pytorch-fid, OpenVINO, random_face)
• Explains conversion from rosinality StyleGAN2, exporting to ONNX/CoreML, demo & evaluation workflows
a. Main components
1. Configuration Manager (JSON + argparse)
2. Data Pipeline (Dataset loader + DiffAug)
3. Model Architectures
– Teacher: StyleGAN2 mapping + synthesis networks (rosinality import)
– Student: MobileStyleGAN mapping + mobile synthesis network
– Discriminator
– Auxiliary: InceptionV3 for perceptual loss
4. Training Engine
– Distiller orchestrates forward/backward passes
– Loss Modules (GAN, perceptual, distill, diffaug)
– Optimizer & LR Scheduler
– Checkpoint Manager
5. Inference & Evaluation
– Image Generator (generate.py)
– FID Evaluator (evaluate_fid.py)
– Demo / Compare Visualizer
6. Conversion & Export
– ROS_IMPORT Module (convert_rosinality_ckpt.py)
– ONNX Exporter
– CoreML Exporter
7. Deployment Wrapper
– random_face + OpenVINO integration
b. Relationships and data flows
• User invokes a script → Config Manager parses JSON & CLI →
• train.py → builds Data Pipeline + Model Architectures → passes data through Teacher & Student → computes losses → updates Student → checkpoints
• generate/demo/compare → loads Student ckpt → samples noise → mapping → synthesis → image output
• evaluate_fid → reads generated & reference images → computes FID via pytorch-fid module
• convert_rosinality_ckpt → loads external ckpt → splits into mapping/synthesis ckpts + writes config JSON
• train.py --export-model → uses PyTorch’s exporter APIs to produce ONNX/CoreML files
• Deployment Wrapper loads exported model → runs inference via OpenVINO
c. Architectural patterns & design principles
• Modular separation: dataset, model definitions, losses, scripts
• Teacher–Student (knowledge distillation) pattern
• Plugin-style custom ops (modules/ops) for performance
• Clear config-driven orchestration to decouple code from hyperparameters
d. Key technologies & frameworks
• PyTorch (core DL framework)
• CUDA & custom C++/CUDA kernels (fused activations, upfirdn2d)
• JSON config files + argparse
• ONNX & CoreML export
• External libs: pytorch-fid, OpenVINO, random_face