How nmtrang/stackgan-roberta works
Project type and purpose
• This is a research/ML pipeline project that reproduces the StackGAN architecture with a RoBERTa‐based text embedding.
• It trains in two sequential GAN stages, reading text captions, embedding them, and generating images.
File structure
• Top‐level directories:
• cfg/ – YAML configuration for stage‐1 and stage‐2 training
• examples/ – sample model outputs (generated images)
• input/ – original reference scripts (char_cnn_rnn_emb.py, roberta_emb.py, dataset utilities)
• src/ – main codebase (entry point, modules for args, dataset, engine, layers, training, utilities)
• Key files:
• src/app.py – CLI entry point
• src/args.py – command‐line/parser definitions
• src/dataset.py – dataset loader & iterator
• src/engine.py – training loop orchestration for both stages
• src/layers.py – model definitions for generator & discriminator
• src/train.py – high‐level train functions for each stage
• src/util.py – helpers (logging, checkpointing, image saving)
• Config & requirements:
• cfg/s1.yml, cfg/s2.yml – hyperparameters per stage
• requirements.txt – Python dependencies (torch, transformers, etc.)
Extract insights from the README
• The system runs in two stages
• It uses pretrained RoBERTa for text embedding (via input/src/roberta_emb.py)
• Outputs (checkpoints & sample images) are stored under examples/ and shared externally
a. Main components
1. Configuration Loader
2. Data Ingestion & Embedding
– raw captions → RoBERTa (or charCNN‐RNN fallback) → text embeddings
3. GAN Stage‐1
– Generator1, Discriminator1
4. GAN Stage‐2
– Generator2, Discriminator2
5. Training Engine
– orchestrates epoch loops, loss computation, optimizer steps, checkpointing
6. Utilities
– Logging, checkpoint I/O, image saving
7. CLI Interface
– arg parsing, stage selection
8. External dependencies
– Pretrained RoBERTa model (HuggingFace Transformers)
– Dataset files (captions + images)
b. Relationships & data flow
– User invokes src/app.py with flags → args.py parses → config loader reads s1.yml or s2.yml
– Dataset module loads caption–image pairs → Embedding module transforms captions → Data batch
– Training engine passes embeddings + noise to Generator → generates fake images → Discriminator compares real vs fake → losses computed → model weights updated
– At end of Stage‐1, checkpoints produced → Stage‐2 engine loads Stage‐1 generator as conditional input, repeats train loop
– Utilities handle periodic logging, checkpoint saving, and sample image dumps → examples/
c. Architectural patterns & principles
– Modular pipeline: clear separation of concerns (data, embedding, model, training, I/O)
– Two‐stage sequential workflow (Stage‐1 → Stage‐2)
– Config‐driven: training hyperparameters and paths injected at runtime
– Reusable embedding layer (RoBERTa) factored out from main code
d. Key technologies/frameworks
– Python 3.x, PyTorch
– HuggingFace Transformers (RoBERTa)
– YAML for configuration
– Standard data‐parallel training loop