# kburdge/gc-variables-pipeline architecture

> Architecture diagram of the GitHub repository kburdge/gc-variables-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kburdge/gc-variables-pipeline
- Repository: https://github.com/kburdge/gc-variables-pipeline

## Overview

This is a JWST/NIRCam time-series photometry pipeline for finding and characterizing variable stars, then rebuilding a vetted catalog. Its main workflow downloads MAST exposures, performs pinned JWST detector calibration, builds high-cadence group-difference cubes, detects variables and extracts light curves, corrects photometry, searches periods, and constructs the catalog from human REAL/FAKE labels. Gaia-tied astrometry supports calibrated coordinates; demo and Segment 1 reproduction are additional capabilities. Several edges follow explicit README documentation; source excerpts show key stage behavior, but not all script wiring.

Main entry points: [`pipeline/detector1.py`](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/detector1.py) (Detector calibration), [`scripts/00_download_mast.py`](https://github.com/kburdge/gc-variables-pipeline/blob/main/scripts/00_download_mast.py) (Exposure download), [`pipeline/extract.py`](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/extract.py) (Extraction orchestration), [`pipeline/astrometry.py`](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/astrometry.py) (Gaia astrometry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_acquisition["Data acquisition"]
  node_download["Exposure download"]
  node_config["Pipeline configuration<br/>[config.py]"]
end

subgraph group_processing["Image processing"]
  node_calibration["Detector calibration<br/>[detector1.py]"]
  node_ramps["Calibrated exposures"]
  node_astrometry["Gaia astrometry<br/>[astrometry.py]"]
  node_cubes["Group-difference cubes<br/>[groupdiff.py]"]
  node_cube_store[("Cube products")]
end

subgraph group_analysis["Variable analysis"]
  node_detection["Autocorrelation detection<br/>[detect.py]"]
  node_extraction["Extraction orchestration<br/>[extract.py]"]
  node_photometry["Aperture photometry<br/>[photometry.py]"]
  node_lightcurves[("Extracted light curves")]
  node_corrections["Photometry corrections<br/>[corrections.py]"]
  node_periods["Period search<br/>[periods.py]"]
end

subgraph group_catalog["Catalog and reproduction"]
  node_vetting["Human REAL/FAKE vetting"]
  node_labels["Vetting labels"]
  node_catalog_build["Catalog construction<br/>[catalog.py]"]
  node_catalog_store[("Published catalog")]
  node_segment1["Segment 1 extraction"]
  node_stage_figures["Segment 1 stage figures"]
end

node_researcher(("Researcher"))
node_mast[("MAST archive")]
node_gaia[("Gaia DR3")]

node_researcher -->|"starts run"| node_download
node_mast -->|"provides exposures"| node_download
node_download -->|"downloads raw"| node_ramps
node_config -->|"configures paths"| node_download
node_config -->|"pins context"| node_calibration
node_ramps -->|"calibrates"| node_calibration
node_calibration -->|"provides ramps"| node_cubes
node_calibration -.->|"provides images"| node_astrometry
node_gaia -->|"provides reference"| node_astrometry
node_astrometry -->|"aligns WCS"| node_cubes
node_cubes -->|"writes cubes"| node_cube_store
node_cube_store -->|"supplies frames"| node_extraction
node_extraction -->|"detects sources"| node_detection
node_extraction -->|"calls photometry"| node_photometry
node_photometry -->|"produces curves"| node_lightcurves
node_lightcurves -->|"corrects flux"| node_corrections
node_corrections -.->|"enables search"| node_periods
node_lightcurves -.->|"searches periods"| node_periods
node_researcher -->|"classifies sources"| node_vetting
node_vetting -->|"exports labels"| node_labels
node_labels -->|"builds from labels"| node_catalog_build
node_catalog_build -->|"writes tables"| node_catalog_store
node_catalog_store -->|"extends catalog"| node_segment1
node_segment1 -->|"supplies stages"| node_stage_figures

click node_download "https://github.com/kburdge/gc-variables-pipeline/blob/main/scripts/00_download_mast.py"
click node_config "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/config.py"
click node_calibration "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/detector1.py"
click node_astrometry "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/astrometry.py"
click node_cubes "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/groupdiff.py"
click node_detection "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/detect.py"
click node_extraction "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/extract.py"
click node_photometry "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/photometry.py"
click node_corrections "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/corrections.py"
click node_periods "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/periods.py"
click node_vetting "https://github.com/kburdge/gc-variables-pipeline/blob/main/scripts/export_vetting_labels.py"
click node_catalog_build "https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/catalog.py"
click node_segment1 "https://github.com/kburdge/gc-variables-pipeline/blob/main/unported/add_segment1_terzan5.py"
click node_stage_figures "https://github.com/kburdge/gc-variables-pipeline/blob/main/unported/paper_seg1_stages.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_download,node_config toneBlue
class node_calibration,node_ramps,node_astrometry,node_cubes,node_cube_store,node_mast,node_gaia toneAmber
class node_detection,node_extraction,node_photometry,node_lightcurves,node_corrections,node_periods toneMint
class node_vetting,node_labels,node_catalog_build,node_catalog_store,node_segment1,node_stage_figures toneRose
class node_researcher toneIndigo
```

## Components

### Data acquisition

- [Exposure download](https://github.com/kburdge/gc-variables-pipeline/blob/main/scripts/00_download_mast.py): `scripts/00_download_mast.py`
- [Pipeline configuration](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/config.py): `pipeline/config.py`

### Image processing

- [Detector calibration](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/detector1.py): `pipeline/detector1.py`
- **Calibrated exposures**
- [Gaia astrometry](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/astrometry.py): `pipeline/astrometry.py`
- [Group-difference cubes](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/groupdiff.py): `pipeline/groupdiff.py`
- **Cube products**

### Variable analysis

- [Autocorrelation detection](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/detect.py): `pipeline/detect.py`
- [Extraction orchestration](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/extract.py): `pipeline/extract.py`
- [Aperture photometry](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/photometry.py): `pipeline/photometry.py`
- **Extracted light curves**
- [Photometry corrections](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/corrections.py): `pipeline/corrections.py`
- [Period search](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/periods.py): `pipeline/periods.py`

### Catalog and reproduction

- [Human REAL/FAKE vetting](https://github.com/kburdge/gc-variables-pipeline/blob/main/scripts/export_vetting_labels.py): `scripts/export_vetting_labels.py`
- **Vetting labels**
- [Catalog construction](https://github.com/kburdge/gc-variables-pipeline/blob/main/pipeline/catalog.py): `pipeline/catalog.py`
- **Published catalog**
- [Segment 1 extraction](https://github.com/kburdge/gc-variables-pipeline/blob/main/unported/add_segment1_terzan5.py): `unported/add_segment1_terzan5.py`
- [Segment 1 stage figures](https://github.com/kburdge/gc-variables-pipeline/blob/main/unported/paper_seg1_stages.py): `unported/paper_seg1_stages.py`

### Other components

- **Researcher**
- **MAST archive**
- **Gaia DR3**

## Connections

- Researcher → Exposure download: starts run (evidence: `README.md`)
- MAST archive → Exposure download: provides exposures (evidence: `README.md`)
- Exposure download → Calibrated exposures: downloads raw (evidence: `README.md`)
- Pipeline configuration → Exposure download: configures paths (evidence: `README.md`)
- Pipeline configuration → Detector calibration: pins context (evidence: `pipeline/config.py`)
- Calibrated exposures → Detector calibration: calibrates (evidence: `README.md`)
- Detector calibration → Group-difference cubes: provides ramps (evidence: `README.md`)
- Detector calibration → Gaia astrometry: provides images
- Gaia DR3 → Gaia astrometry: provides reference (evidence: `pipeline/astrometry.py`)
- Gaia astrometry → Group-difference cubes: aligns WCS (evidence: `pipeline/groupdiff.py`)
- Group-difference cubes → Cube products: writes cubes (evidence: `README.md`)
- Cube products → Extraction orchestration: supplies frames (evidence: `pipeline/extract.py`)
- Extraction orchestration → Autocorrelation detection: detects sources (evidence: `pipeline/extract.py`)
- Extraction orchestration → Aperture photometry: calls photometry (evidence: `pipeline/extract.py`)
- Aperture photometry → Extracted light curves: produces curves
- Extracted light curves → Photometry corrections: corrects flux (evidence: `README.md`)
- Photometry corrections → Period search: enables search
- Extracted light curves → Period search: searches periods
- Researcher → Human REAL/FAKE vetting: classifies sources (evidence: `scripts/export_vetting_labels.py`)
- Human REAL/FAKE vetting → Vetting labels: exports labels (evidence: `scripts/export_vetting_labels.py`)
- Vetting labels → Catalog construction: builds from labels (evidence: `pipeline/catalog.py`)
- Catalog construction → Published catalog: writes tables (evidence: `pipeline/catalog.py`)
- Published catalog → Segment 1 extraction: extends catalog (evidence: `unported/add_segment1_terzan5.py`)
- Segment 1 extraction → Segment 1 stage figures: supplies stages (evidence: `unported/paper_seg1_stages.py`)
