How eric-yyjau/pytorch-superpoint works
Project Type and Purpose
• This repository is a research‐oriented PyTorch implementation of the SuperPoint keypoint detector+descriptor network.
• It provides a full deep learning pipeline: synthetic data generation → MagicPoint detector training → homography‐adaptation label export → joint SuperPoint training → descriptor/keypoint export and evaluation.
File Structure Analysis
• Top‐level scripts:
• train4.py, Train_model_*.py, Val_model_*.py – wrappers to start specific training/validation jobs
• export.py, export_classical.py – pipelines to generate pseudo‐ground‐truth and classical‐detector outputs
• evaluation.py – computes metrics (repeatability, matching score, homography estimation)
• run_export.sh – batch export+evaluate for HPatches
• Directories:
• configs/ – YAML files defining tasks, dataset roots, model hyperparameters
• datasets/ – Dataset reader classes (Coco, Kitti, Tum, Synthetic), splitting files, raw KITTI data
• models/ – PyTorch model definitions (SuperPointNet, SubpixelNet, U-Net blocks, classical detectors)
• utils/ – Helpers (augmentation, homographies, losses, logging)
• evaluations/ – High‐level evaluation routines for detectors and descriptors
• logs/ – Checkpoints, config.yml, predictions for each experiment
• notebooks/ – Interactive exploration and visualization
• pretrained/ – Downloaded SuperPoint v1 weights
• test/ – Unit tests / small example scripts
README Insights
• Details a 5‐stage pipeline:
2. Export homography‐adapted labels on COCO/KITTI/TUM
3. Train SuperPoint on pseudo‐labels
4. Export+evaluate on HPatches
5. Export+evaluate SIFT baseline
• Dependencies: PyTorch, torchvision, YAML configs, CUDA
• External datasets: MS-COCO, KITTI raw, HPatches
Main Components & Interactions
a. Components (group into functional layers):
1. Configuration Manager (configs/*.yaml + settings.py)
2. Data Layer
• Synthetic Data Generator (datasets/synthetic_shapes.py)
• Dataset Loaders (datasets/Coco.py, Kitti_inh.py, Tum.py)
• Data Augmenters (utils/photometric_augmentation.py, dataset utils)
3. Model Layer
• MagicPoint Detector (models/SuperPointNet*.py, classical_detectors_descriptors.py)
• Subpixel Refinement Net (models/SubpixelNet.py)
• Joint Descriptor Head (models/model_wrap.py)
4. Training & Export Pipelines
• train4.py → calls a Trainer class (uses models, datasets, utils) → logs checkpoints
• export.py → loads checkpoint → runs homography adaptation → writes .npz pseudo‐labels
5. Evaluation Layer
• evaluation.py + evaluations/*.py compute metrics on .npz outputs
6. Visualization & Notebooks (tensorboard via utils/logging.py, notebooks/*.ipynb)
7. Storage & Artifacts
• Dataset files (raw/augmented)
• Checkpoints & logs (logs/*)
• Exported labels & predictions (logs/*/predictions/*.npz)
• Pretrained weights (pretrained/)
b. Relationships & Data Flow
1. Configuration YAML → Pipeline Launcher (train4.py / export.py)
2. Pipeline → Data Layer → Model Layer (forward/backward in training or inference)
3. Model Layer → Checkpoint Writer → logs/<experiment>/checkpoints
4. Export Pipeline → loads checkpoint → applies homography adaptation → writes labels/predictions
5. Evaluation Pipeline → reads predictions → computes metrics → outputs plots/text
c. Architectural Patterns & Principles
• Modular design: each stage (train, export, eval) separated into its own script
• Clear separation of concerns: configs manage hyperparams, datasets isolate I/O, models define networks, utils abstract common code
• Pipeline orchestration: command‐line entry points that glue together config + modules
d. Key Technologies & Frameworks
• PyTorch for model definition and training
• YAML for configuration
• NumPy/.npz for label storage
• TensorBoard for logging
• Jupyter notebooks for interactive analysis