How geragzgm/modelzoo works
Project Type and Purpose
• It is an open-source CLI application for standardized training and inference of CNN architectures (AlexNet, ResNet, VGG, MobileNet, etc.).
• It uses JSON configs to drive experiments, supports multiple dataset backends (CIFAR-10, Kaggle), and provides reproducible checkpointing.
File Structure
• Top level:
• _configs/ contains JSON files for each model.
• modelzoo/ Python package.
• pyproject.toml & setup.py for packaging.
• Inside modelzoo/:
• cli.py, main.py – CLI entry points.
• build_config.py – loads and validates JSON.
• dataset_loaders/ – abstracts data ingestion.
• models/ – plugin directories per model, all inheriting from base_models.py.
• train/ – training/inference logic in base_train.py, inference.py, and classification routines.
• utils.py – shared utilities.
• tests/ – unit tests (e.g., registry tests).
README for Architectural Insights
• Config-driven: everything from optimizer to transforms is JSON-specified.
• Extensible: adding a new model means dropping its folder into models/ and updating config.
• CLI patterns: commands (--run_type train or inference) orchestrate workflows.
Main Components
a. CLI Layer
– modelzoo shell command → main.py & cli.py
– Parses args, chooses “train” vs. “inference.”
b. Configuration Loader
– build_config.py reads _configs/*.json
– Produces a typed config object.
c. Dataset Loader
– dataset_loaders/base_dataset.py and concrete loaders.
– Interfaces with local CIFAR-10 or Kaggle API.
d. Model Registry & Factory
– models/base_models.py + per-model model.py
– Registry pattern: cli asks for “VGG16” → factory returns the class.
e. Training & Inference Engine
– train/base_train.py: orchestrates epochs, batching, checkpointing.
– train/inference.py: runs forward passes, outputs metrics.
– train/src/classification.py: loss, metrics, callbacks.
f. Checkpoint & Artifacts
– Checkpoints, logs written under models/<ModelName>/chkpt/.
g. External Services & Data
– Kaggle API for dataset download.
– File system for config and model saving.
h. Testing
– tests/test_registers.py: ensure model registration integrity.
Relationships & Interactions
• Bidirectional: Config Loader ↔ Model Factory (to match model names).
• Data flow: Raw Data → Transforms → Batches → Model → Loss → Optimizer.
Architectural Patterns & Principles
• Plugin/Registry Pattern for models & dataset loaders.
• Configuration-Driven Design: all parameters decoupled from code.
• Separation of Concerns: CLI, config parsing, data loading, model definitions, training logic all isolated in their own modules.
• Reproducibility via checkpointing and seedable loaders.
Technologies & Frameworks
• Python 3 (packaged via setup.py / Poetry).
• Likely PyTorch (implied by .pth checkpoints).
• Kaggle API for external data.
• JSON for configs.
Detail Level
• Python modules as subcomponents where relevant.
• Methods:
• build_config.load()
• DatasetLoader.get_dataloader()
• ModelFactory.create_model()
• BaseTrain.train_epoch() / Inference.run()