How harry-kit/image-classification-on-small-datasets-in-pytorch works
Project type and purpose
• This is a PyTorch-based research toolkit (a CLI application) for fine-tuning and feature-extracting a variety of pretrained CNN and Vision Transformer models on small image datasets.
• It provides utilities to split a dataset, train models, evaluate metrics, and generate Grad-CAM visualizations (static and video).
File structure
• Top-level scripts (CLI entry points):
• split_dataset.py
• train.py
• test.py
• grad_cam_results.py
• grad_cam_video_results.py
• Core modules:
• config.py – experiment hyperparameters (paths, image size, batch size, etc.)
• data_loader.py – DataLoader wrapper for train/val image directories
• cnn_models.py – factory for initializing torchvision models with custom heads
• metrics.py – functions to compute accuracy, loss curves, confusion matrices, classification reports
• utils.py – plotting utilities, checkpoint save/load, seeding, device management
• Folders for I/O:
• samples1/ – example input images and Grad-CAM video/gif samples
• examples/ – output folders per model, containing accuracy/loss plots, confusion matrices, Grad-CAM images
README for additional insights
• The README describes the four CLI steps:
a. split_dataset.py – organize raw images into train/val subfolders
b. train.py – choose a model, fine-tune or feature-extract, save .pth checkpoint and metric plots in examples/[model]-gradresults/
c. grad_cam_results.py & grad_cam_video_results.py – load checkpoint + input image, run Grad-CAM, write visualizations back to an examples folder
d. test.py – likely loads a checkpoint and runs on a test set (not detailed but implied)
• The architecture is entirely local; no external services, databases, or servers are involved.
Main components
a. CLI Layer
• Scripts: split_dataset, train, test, grad_cam_results, grad_cam_video_results
b. Core Library Layer
• Config Module
• DataLoader Module
• Model Factory Module
• Metrics Module
• Utils Module
c. External Library Layer
• PyTorch / torchvision.models
• OpenCV or PIL (for image I/O, plotting)
• Matplotlib or Seaborn (for plots)
d. Data Storage
• Raw Dataset Directory (input)
• Organized Train/Val directories (after split)
• Checkpoints (.pth)
• Plots & Grad-CAM images/videos under examples/ and samples1/
Relationships and data flows
• split_dataset.py reads Raw Dataset → writes Train/Val folders
• train.py
• imports config, data_loader → loads images → loads a chosen CNN model via cnn_models.initialize_model → trains → writes checkpoint + metrics to disk
• test.py
• loads checkpoint → runs evaluation on dataset → possibly writes additional metrics
• grad_cam_results.py & grad_cam_video_results.py
• load checkpoint + a sample image → forward pass + Grad-CAM hook via utils or dedicated GradCAM implementation → write overlays/videos
Architectural patterns & design principles
• Modular design: clear separation between data loading, model initialization, training logic, and visualization.
• Factory pattern in cnn_models.py to decouple model selection from training logic.
• Single-responsibility modules: each file handles one concern.
• Use of configuration file (config.py) for hyperparameters centralization.