# rpautrat/superpoint architecture

> Architecture diagram of the GitHub repository rpautrat/superpoint, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rpautrat/superpoint
- Repository: https://github.com/rpautrat/superpoint

## Overview

##### Project Type and Purpose
- This is a research‐focused deep learning pipeline (Python package) implementing the “SuperPoint” interest point detector & descriptor.
- It supports both TensorFlow (original) and PyTorch (conversion) backends, end‐to‐end training on synthetic & real datasets, exporting predictions, and evaluation (repeatability, homography estimation).

##### File Structure Analysis
- Top level: configuration & setup (`setup.py`, `requirements.txt`, `makefile`), documentation (`doc/`), Jupyter notebooks (`notebooks/`), pretrained weights (`pretrained_models/`, `weights/`), and the main package directory `superpoint/`.
- `superpoint/` subdivides into:
- `configs/` – all YAML files driving data paths, hyperparameters, augmentation, and model settings
- `models/` – network definitions (MagicPoint, SuperPoint, classical baselines, backbone modules)
- `evaluations/` – scripts to compute detector repeatability and descriptor homography estimation
- Export & control scripts at package root: `experiment.py` (train/eval entry point), `export_detections.py`, `export_descriptors.py`, etc.
- Utilities: logging, bitsets, stdout capturing, cluster environment scripts.

##### Key README Insights
- The pipeline is entirely config‐driven: users modify a YAML in `superpoint/configs/` and invoke `experiment.py` or one of the export scripts.
- Data flows: raw images → synthetic generation/homographic adaptation → model training → predictions → evaluation notebooks.
- Multi‐stage: train MagicPoint on synthetic, export pseudo‐GT on COCO, train SuperPoint, evaluate on HPatches.

##### Main Components
A. Data Sources
1. External datasets: MS‐COCO, HPatches (on disk under `$DATA_DIR`)
B. Data Processing Pipeline
– Augmentation & homography code (in `datasets/utils/` and `models/homographies.py`)
– Creates image pairs / patches for training and evaluation
C. Model Training
1. MagicPoint model (defined in `models/magic_point.py`)
2. SuperPoint model (defined in `models/super_point.py`)
3. Training driver: `experiment.py` (reads a YAML config, launches multi‐GPU training, writes weights & TensorBoard logs to `$EXPER_DIR`)
D. Export & Evaluation
1. Export detections: `export_detections.py` / `export_detections_repeatability.py`
2. Export descriptors: `export_descriptors.py`
3. Evaluation scripts: `evaluations/descriptor_evaluation.py`, `evaluations/detector_evaluation.py`
4. Visualization & result analysis via Jupyter notebooks
E. Utilities & External Integrations
– Configuration (YAML files)
– Logging & cluster environment (`utils/env_cluster.sh`)
– TensorBoard for metrics & visualizations
– Pretrained model loader (`superpoint_pytorch.py` for PyTorch conversion)

##### Relationships & Data Flow
- From Checkpoints → Export scripts → Prediction files
- From Predictions + Ground‐Truth → Evaluation modules → Metrics & Plots → Notebooks
- Config files feed into both training and export scripts
- Pretrained weights can enter the Training stage as inputs

##### Architectural Patterns & Design Principles
- Modular, config‐driven pipeline: all behaviors controlled by YAML in `configs/`
- Clear separation: Data (datasets/) ↔ Models (models/) ↔ Controllers (`experiment.py`, export scripts) ↔ Evaluation (evaluations/, notebooks/)
- Extensibility points: add new dataset under `datasets/`, new model under `models/`, new evaluation under `evaluations/`
- Reproducibility: experiment directory outputs everything (weights, logs, predictions)

##### Technologies, Frameworks, Libraries
- Python 3.6+, TensorFlow (original), PyTorch (conversion)
- YAML for configuration
- NumPy, OpenCV for image handling and augmentation
- MS‐COCO and HPatches external datasets
- Jupyter notebooks for interactive evaluation
- TensorBoard for monitoring

##### Elements
- Config panel showing YAML files driving behavior
- Data stores (COCO, HPatches, synthetic)
- Pretrained model input path and TensorBoard logging integration

##### Final Advice
- Lifecycle: raw data → training → checkpoints → prediction → metrics.
- Don’t hesitate to break each pipeline stage into sub‐components to reveal all configuration and augmentation steps.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources"
        SS["Synthetic Shapes Generator"]:::data
        COCO["COCO Dataset Loader"]:::data
        HP["HPatches & Patch Dataset"]:::data
        MN["MNIST Loader"]:::data
    end

    %% Data Processing
    subgraph "Data Processing"
        PA["Photometric Augmentation"]:::process
        AL["Legacy Augmentation Utilities"]:::process
        DP["Dataset Pipeline Assembly"]:::process
        HA["Homographic Adaptation Code"]:::process
    end

    %% Model Training
    subgraph "Model Training"
        CFG["Configuration Files (YAML)"]:::config
        MP["MagicPoint Model"]:::train
        SP["SuperPoint Model"]:::train
        EXP["Experiment / Training Driver"]:::train
    end

    %% Export & Evaluation
    subgraph "Export & Evaluation"
        ED["export_detections.py"]:::export
        ER["export_detections_repeatability.py"]:::export
        EDESC["export_descriptors.py"]:::export
        EM["export_model.py"]:::export
        DEVAL["descriptor_evaluation.py"]:::export
        REVAL["detector_evaluation.py"]:::export
        NB["Notebooks (Visualization & Analysis)"]:::export
    end

    %% Utilities
    subgraph "Utilities & Integrations"
        PT["PyTorch Conversion Entry Point"]:::util
        CL["Cluster Environment Setup"]:::util
        LOG["Logging & Stdout Capturing"]:::util
        BS["Bitset & Helper Tools"]:::util
        TB["TensorBoard"]:::util
        PW["Pretrained Weights Storage"]:::util
    end

    %% Data Flow
    SS -->|raw images| PA
    COCO -->|datasets| DP
    HP --> DP
    MN --> DP
    PA --> DP
    AL --> DP
    HA --> DP

    DP -->|batches| EXP
    CFG --> EXP
    PT --> EXP

    EXP -->|checkpoints| MP
    EXP --> SP
    MP --> SP

    MP -->|weights| EM
    SP -->|weights| EM

    EM -->|converted model| PT
    EM --> ED
    EM --> ER
    EM --> EDESC

    ED --> DEVAL
    ER --> REVAL
    EDESC --> DEVAL
    DEVAL --> NB
    REVAL --> NB

    EXP -->|logs| TB

    CL --> EXP
    LOG --> EXP
    BS --> EXP

    PW --> EXP

    %% Click Events
    click SS "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/synthetic_shapes.py"
    click COCO "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/coco.py"
    click HP "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/patches_dataset.py"
    click MN "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/mnist.py"
    click PA "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/utils/photometric_augmentation.py"
    click AL "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/utils/augmentation_legacy.py"
    click DP "https://github.com/rpautrat/superpoint/blob/master/superpoint/datasets/utils/pipeline.py"
    click HA "https://github.com/rpautrat/superpoint/blob/master/superpoint/models/homographies.py"
    click MP "https://github.com/rpautrat/superpoint/blob/master/superpoint/models/magic_point.py"
    click SP "https://github.com/rpautrat/superpoint/blob/master/superpoint/models/super_point.py"
    click EXP "https://github.com/rpautrat/superpoint/blob/master/superpoint/experiment.py"
    click CFG "https://github.com/rpautrat/superpoint/tree/master/superpoint/configs/"
    click ED "https://github.com/rpautrat/superpoint/blob/master/superpoint/export_detections.py"
    click ER "https://github.com/rpautrat/superpoint/blob/master/superpoint/export_detections_repeatability.py"
    click EDESC "https://github.com/rpautrat/superpoint/blob/master/superpoint/export_descriptors.py"
    click EM "https://github.com/rpautrat/superpoint/blob/master/superpoint/export_model.py"
    click DEVAL "https://github.com/rpautrat/superpoint/blob/master/superpoint/evaluations/descriptor_evaluation.py"
    click REVAL "https://github.com/rpautrat/superpoint/blob/master/superpoint/evaluations/detector_evaluation.py"
    click NB "https://github.com/rpautrat/superpoint/tree/master/notebooks/"
    click PT "https://github.com/rpautrat/superpoint/blob/master/superpoint_pytorch.py"
    click CL "https://github.com/rpautrat/superpoint/blob/master/utils/env_cluster.sh"
    click LOG "https://github.com/rpautrat/superpoint/blob/master/superpoint/utils/stdout_capturing.py"
    click BS "https://github.com/rpautrat/superpoint/blob/master/superpoint/utils/bitset.py"
    click TB "https://github.com/rpautrat/superpoint/tree/master/pretrained_models/"
    click PW "https://github.com/rpautrat/superpoint/tree/master/weights/"

    %% Styles
    classDef data fill:#FFD7D7,stroke:#E64545,shape:cylinder
    classDef process fill:#FFF5C3,stroke:#D7B740,shape:hexagon
    classDef train fill:#C3DFFF,stroke:#3B82F6,shape:round-rect
    classDef export fill:#D7FFD9,stroke:#22C55E,shape:parallelogram
    classDef util fill:#EDEDED,stroke:#6B7280,shape:cloud
    classDef config fill:#F0F5FF,stroke:#6366F1,shape:gear
```
