How sumyeongahn/cuda_ltr works
Main Components
• Data Input & Dataset Modules:
– For CIFAR, look at the “cifar/datasets” folder (e.g., cifar100.py) which handles loading and serving CIFAR-100 data.
– For large-scale use cases, the “large_scale” directory includes configuration files, imbalance_data routines, and dataset-specific scripts.
• Data Augmentation:
– “cifar/aug” and “large_scale/aug” provide various augmentation methods (e.g., autoaug, cuda, cutmix/cutout, randaug, transforms).
– These modules are key in applying the CUDA curriculum (per-class augmentation strength) and other augmentation policies.
• Loss Functions:
– Under “cifar/losses”, multiple loss functions are defined (e.g., balanced contrastive loss, cross-entropy variants, LDAM-based losses, NCL, RIDE).
– For large-scale experiments, there is a “large_scale/losses.py” file.
• Model Architectures:
– The “cifar/models” folder houses definitions such as “net.py” and different variations of resnet (plain, with BCL, with NCL, with RIDE).
– These represent the backbone networks and any tailored modifications in the training pipelines.
• Training Pipeline & Functions:
– Separate training routines exist in both major directories:
• For CIFAR-100: “cifar/main.py” is the training entry point and “cifar/train” includes training functions, switching among different loss functions or augmentation strategies.
• For large-scale datasets: “large_scale/train.py” and associated helper files (like “training_functions.py” and dataset-specific training scripts such as “imagenet_train.py”, “inat18_train.py”).
• Utilities & Configuration:
– Both directories have directories for configuration (cifar/utils/config.py, large_scale/config) and additional utilities (e.g., logger, accuracy, plotting) that support debugging and evaluating performance.
– The “requirements.txt” file specifies all external dependencies (likely PyTorch and others).
Relationships and Interactions
• Data flows into the Data Augmentation modules where various augmentation strategies, including the CUDA curriculum, are applied.
• After augmentation, data is fed into the Model Architectures. Training scripts from “cifar/main.py” or “large_scale/train.py” coordinate this process.
• The Model’s output is then processed by the Loss Functions for backpropagation. This loss is computed using modules from the “losses” directories.
• Finally, training control and evaluation occur via the Training Pipelines combined with utility functions (logger, accuracy, config, and plotting) for monitoring progress and visualization of results.
• In addition, consider showing how different training configurations are “swapped” or toggled based on experimental settings (e.g., using DRW, CE, or other loss/augmentation modes).
Architectural Patterns & Design Principles
• Separation of Concerns: Data loading, augmentation, modeling, and training are clearly separated into distinct directories.
• Modularity: Both the CIFAR and large-scale pipelines are modular – loss functions, training strategies, and augmentation methods are clearly encapsulated, allowing for easy experimentation.
• Configurability: Through configuration files and command-line options, the project allows switching between various augmentation and training policies (e.g., cutout vs CUDA augmentation).
Relevant Technologies/Frameworks
• Deep Learning Framework: Likely based on PyTorch (implied by the Python training scripts and prevalent deep learning practices).
• Jupyter Notebooks: Seen in “cifar/analysis/fix_fe.ipynb” for analysis or debugging.
• Standard data augmentation libraries, custom augmentation modules, and potentially external libraries for image processing.