# mwalmsley/zoobot-3d architecture

> Architecture diagram of the GitHub repository mwalmsley/zoobot-3d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mwalmsley/zoobot-3d
- Repository: https://github.com/mwalmsley/zoobot-3d

## Overview

##### Project Type and Purpose
- This is an end-to-end machine‐learning research pipeline for galaxy‐image segmentation and (optionally) classification.
- It ingests volunteer‐annotated FITS data (GZ3D), DESI‐LS cutouts, and produces trained UNet models and evaluation artifacts (figures).

##### Top-Level Components
A. External Data Sources
– GZ3D FITS service (raw volunteer marks, segmentation maps, MANGA metadata)
– DESI‐LS cutout API/service
– SAMI data‐central service
– GZ DESI and GZ2 volunteer‐vote tables (parquet)
B. Data Preparation Layer (`prepare_data/`, `deprecated/`, notebooks)
– FITS scraping & download (`download_gz3d_fits.py`, `download_centered_desi_cutouts.py`)
– Vector‐to‐mask extraction (`extract_gz3d_segmaps.py` → `zoobot_3d.segmap_utils.construct_segmap_image`)
– Catalog construction & crossmatch (`construct_gz3d_catalog.py`, `crossmatch_catalogs.ipynb`)
– Final catalog tweaks (`final_catalog_tweaks.ipynb`)
– Auxiliary scripts (sparcfire, custom legacy)
C. Configuration (`conf/config.yaml`, `setup.py`)
D. Data Storage
– Local file hierarchy under `data/` (FITS, JSON segmap, JPGs, parquet catalogs)
E. Data‐Loading & Augmentation (`zoobot_3d/pytorch_dataset.py`, `pytorch_datamodule.py`)
– LightningDataModule managing train/val/test splits, optional mask/class labels, albumentations transforms
F. Model Definition (`zoobot_3d/gz3d_pytorch_model.py`, `custom_layers.py`)
– UNet encoder/decoder, optional classification head, Dirichlet loss support
G. Training & Hyperparameter Sweep
– `train.py`: orchestrates training on spiral/bar segmaps only
– Slurm job scripts (`slurm/train.sh`, `slurm/sweep.yaml`, `run_sweep_agent.sh`, `run_experiments.sh`)
H. Prediction & Evaluation
– `predict.py`: batch inference over new galaxies
– Results notebooks (`results_for_neurips/*.ipynb`) generate comparison grids and expert‐vote analyses
I. Outputs
– Model checkpoints, metrics logs, paper‐ready figures under `outputs/`
J. CI / Packaging
– `setup.py` for pip install, `requirements.txt` (implicit)

##### Interactions and Data Flow
- External Data Services → Data Preparation Layer → Data Storage
- Data Storage → Data‐Loading Layer → Model
- Model + DataLoader → Training Engine (`train.py` via Slurm) → Outputs (checkpoints, logs)
- Checkpoints → Prediction Script (`predict.py`) → Evaluation Notebooks → Final Figures
- Config (`conf/config.yaml`) guides dataset paths, model/hparam defaults

##### Architectural Patterns & Principles
- Separation of concerns: Data ingestion, preprocessing, modeling, and evaluation are modular.
- Pipeline pattern: clear linear flow from raw data to figures, with branching for sweeps and predictions.
- HPC‐friendly design: Slurm scripts wrap Python entry points, enabling reproducible sweeps.
- Config‐driven: YAML config + environment variables allow swapping datasets or hyperparams.
- PyTorch Lightning (via DataModule) for standardized training loops.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Services
    subgraph "External Data Services"
        direction TB
        GZ3D["GZ3D FITS Service"]:::external
        DESI["DESI-LS Cutout API"]:::external
        SAMI["SAMI Data Service"]:::external
        VOTES["Volunteer Vote Tables (parquet)"]:::external
    end

    %% Preprocessing Layer
    subgraph "Data Preparation (prepare_data)" 
        direction TB
        DownloadFITS["download_gz3d_fits.py"]:::compute
        click DownloadFITS "https://github.com/mwalmsley/zoobot-3d/blob/main/prepare_data/download_gz3d_fits.py"
        DownloadDESI["download_centered_desi_cutouts.py"]:::compute
        click DownloadDESI "https://github.com/mwalmsley/zoobot-3d/blob/main/prepare_data/download_centered_desi_cutouts.py"
        DESIShell["download_desi_cutouts.sh"]:::compute
        click DESIShell "https://github.com/mwalmsley/zoobot-3d/blob/main/slurm/download_desi_cutouts.sh"
        ExtractMask["extract_gz3d_segmaps.py"]:::compute
        click ExtractMask "https://github.com/mwalmsley/zoobot-3d/blob/main/prepare_data/extract_gz3d_segmaps.py"
        SegmapUtils["segmap_utils.construct_segmap_image"]:::compute
        click SegmapUtils "https://github.com/mwalmsley/zoobot-3d/blob/main/zoobot_3d/segmap_utils.py"
        ConstructCat["construct_gz3d_catalog.py"]:::compute
        click ConstructCat "https://github.com/mwalmsley/zoobot-3d/blob/main/prepare_data/construct_gz3d_catalog.py"
        Crossmatch["crossmatch_catalogs.ipynb"]:::notebook
        click Crossmatch "https://github.com/mwalmsley/zoobot-3d/blob/main/prepare_data/crossmatch_catalogs.ipynb"
        FinalTweak["final_catalog_tweaks.ipynb"]:::notebook
        click FinalTweak "https://github.com/mwalmsley/zoobot-3d/blob/main/prepare_data/final_catalog_tweaks.ipynb"
    end

    %% Configuration
    Config["conf/config.yaml"]:::config
    click Config "https://github.com/mwalmsley/zoobot-3d/blob/main/conf/config.yaml"
    Setup["setup.py"]:::config
    click Setup "https://github.com/mwalmsley/zoobot-3d/blob/main/setup.py"

    %% Data Store
    DataStore["Data Store (FITS/JSON/JPG/Parquet)"]:::storage

    %% Data Loading & Augmentation
    subgraph "Data Loading & Augmentation"
        direction TB
        Dataset["pytorch_dataset.py"]:::compute
        click Dataset "https://github.com/mwalmsley/zoobot-3d/blob/main/zoobot_3d/pytorch_dataset.py"
        DataModule["pytorch_datamodule.py"]:::compute
        click DataModule "https://github.com/mwalmsley/zoobot-3d/blob/main/zoobot_3d/pytorch_datamodule.py"
    end

    %% Model Definition
    subgraph "Model Definition"
        direction TB
        ModelDef["gz3d_pytorch_model.py"]:::compute
        click ModelDef "https://github.com/mwalmsley/zoobot-3d/blob/main/zoobot_3d/gz3d_pytorch_model.py"
        CustomLayers["custom_layers.py"]:::compute
        click CustomLayers "https://github.com/mwalmsley/zoobot-3d/blob/main/zoobot_3d/custom_layers.py"
    end

    %% Training & Hyperparam Sweep
    subgraph "Training & HPC Jobs"
        direction TB
        Trainer["train.py"]:::compute
        click Trainer "https://github.com/mwalmsley/zoobot-3d/blob/main/train.py"
        SlurmScript["train.sh"]:::compute
        click SlurmScript "https://github.com/mwalmsley/zoobot-3d/blob/main/slurm/train.sh"
        SweepConfig["sweep.yaml"]:::compute
        click SweepConfig "https://github.com/mwalmsley/zoobot-3d/blob/main/slurm/sweep.yaml"
        SweepAgent["start_sweep_agent.sh"]:::compute
        click SweepAgent "https://github.com/mwalmsley/zoobot-3d/blob/main/slurm/start_sweep_agent.sh"
        RunExp["run_experiments.sh"]:::compute
        click RunExp "https://github.com/mwalmsley/zoobot-3d/blob/main/slurm/run_experiments.sh"
    end

    %% Prediction & Evaluation
    Predictor["predict.py"]:::compute
    click Predictor "https://github.com/mwalmsley/zoobot-3d/blob/main/predict.py"
    subgraph "Evaluation Notebooks"
        direction TB
        CompareGrid["create_comparison_grids.ipynb"]:::notebook
        click CompareGrid "https://github.com/mwalmsley/zoobot-3d/blob/main/results_for_neurips/create_comparison_grids.ipynb"
        ExpertVotes["expert_votes.ipynb"]:::notebook
        click ExpertVotes "https://github.com/mwalmsley/zoobot-3d/blob/main/results_for_neurips/expert_votes.ipynb"
    end

    %% Outputs
    Outputs["Model Checkpoints & Figures"]:::storage

    %% Arrows & Data Flow
    GZ3D -->|FITS| DownloadFITS
    DESI -->|HTTP| DownloadDESI
    DESI -->|rsync| DESIShell
    DownloadFITS --> ExtractMask
    ExtractMask --> SegmapUtils
    SegmapUtils --> ConstructCat
    ConstructCat --> Crossmatch
    Crossmatch --> FinalTweak
    FinalTweak --> DataStore
    Config --> DownloadFITS
    Config --> Trainer
    Config --> Dataset

    DataStore --> Dataset
    Dataset --> DataModule
    DataModule --> ModelDef
    ModelDef --> Trainer
    CustomLayers --> Trainer

    Trainer --> Outputs
    Trainer --> SweepConfig
    SweepConfig --> SweepAgent
    SweepAgent --> RunExp
    RunExp --> Outputs

    Outputs --> Predictor
    Predictor --> CompareGrid
    Predictor --> ExpertVotes

    %% Styles
    classDef external fill:#bbdefb,stroke:#1e88e5;
    classDef storage fill:#c8e6c9,stroke:#43a047,stroke-width:2px;
    classDef compute fill:#ffe0b2,stroke:#fb8c00;
    classDef notebook fill:#e1bee7,stroke:#8e24aa,stroke-width:2px;
    classDef config fill:#eceff1,stroke:#607d8b;
```
