# kvos/coastsat architecture

> Architecture diagram of the GitHub repository kvos/coastsat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kvos/coastsat
- Repository: https://github.com/kvos/coastsat

## Overview

##### Project type and purpose
- Project type: Open‑source Python toolkit / library (no web UI) for automated shoreline detection, time‑series analysis and tidal correction.
- Key goal: Download satellite imagery (Landsat, Sentinel‑2) → preprocess → classify pixels → extract shorelines → intersect with transects → apply tidal correction & slope estimation → output results (GeoJSON, pickles, plots).

##### File structure
- Top‑level modules:
– coastsat/ : core Python package modules (SDS_download, SDS_preprocess, SDS_classify, SDS_shoreline, SDS_transects, SDS_tools)
– classification/ : training data, Jupyter notebook, ML models (.pkl)
– examples/ & example.py/ipynb: demonstration scripts and sample data
– doc/ : documentation & setup guides
- Config & CI: .github/workflows/test‑coastsat.yml for tests, dependencies in README.
- Outputs: .geojson, .pkl, .mp4, figures saved to user‐specified “filepath/sitename” directory.

##### README insights
- Describes six functional blocks: retrieval, detection, transects intersection, tidal correction & slope, postprocessing, validation.
- Lists external dependencies: Google Earth Engine API, FES2022 tide model, GDAL, geopandas, scikit‑image, PyFES, etc.

1. User Interface (CLI or notebooks/scripts)
2. Image Retrieval Module (SDS_download → calls Google Earth Engine API)
3. Preprocessing Module (SDS_preprocess → reprojection, cloud masking, pansharpening)
4. Classification Module (ML models loaded from classification/models/*.pkl → pixel labelling into sand/water/whitewater/other)
5. Shoreline Extraction Module (SDS_shoreline → segmentation + thresholding → polyline generation)
7. Tidal Correction & Slope Estimation (SDS_transects + external FES2022 service or CSV tide inputs + CoastSat.slope library)
8. Post‑Processing & Analysis (despiking, seasonal/monthly averages, trend estimation)
9. Output & Visualization (GeoJSON exports, .pkl pickles, plots, timelapse MP4 via SDS_tools)
10. Data Storage (local filesystem: imagery, intermediate files, outputs)
11. External Services:
– Google Earth Engine
– Global tide model (FES2022)
- Image Retrieval → writes → Raw imagery files → Preprocessing Module
- Preprocessing → writes → Preprocessed imagery → Classification Module
- Classification → writes → Classified raster → Shoreline Extraction Module
- Shoreline Extraction → writes → Shorelines (GeoJSON + Python objects) → Transect Module
- Transect Module ↔ Tidal Correction & Slope Estimation (requires tide time‑series + slope)
- Tidal Correction & Slope → writes → Corrected shoreline time‑series → Post‑Processing & Analysis
- Post‑Processing → writes → Final outputs (GeoJSON, .pkl, figures)
- Transect Module & Post‑Processing → may call → SDS_tools for animations, plots
c. Architectural patterns & design principles:
- Modular pipeline: clear separation of concerns between retrieval, preprocessing, analysis, output.
- Plug‑in ML classifiers: .pkl files under classification/models can be swapped or retrained.
- Extensibility points: add new cloud‑mask algorithms, new classifiers, alternate tidal models.
- Data‑centric: each stage reads/writes files on disk to maximize reproducibility and checkpointing.
- Python 3.11, Earth Engine API, GDAL, geopandas, shapely, scikit‑image, numpy, matplotlib
- PyFES for tide predictions, imageio‑ffmpeg for animations
- Jupyter Notebook / Spyder for user interaction
- GitHub Actions for CI testing

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface"
        style UI fill:#EFEFFF,stroke:#333,stroke-width:1px
        UI1("example.py"):::ui
        UI2("example_jupyter.ipynb"):::ui
        UI3("test.py"):::ui
    end

    %% Core Processing Modules
    subgraph "Retrieval Module"
        style Retrieval fill:none,stroke:none
        Download["SDS_download: Image Retrieval"]:::process
        EarthEngine("Google Earth Engine"):::external
    end

    subgraph "Preprocessing Module"
        style Preprocessing fill:none,stroke:none
        Preprocess["SDS_preprocess: Reprojection, Cloud Masking, Pansharpening"]:::process
    end

    subgraph "Classification Module"
        style Classification fill:none,stroke:none
        Classify["SDS_classify: Pixel Labelling"]:::process
        Models["classification/models/*.pkl"]:::datastore
        TrainNB("classification/train_new_classifier.ipynb"):::ui
    end

    subgraph "Shoreline Extraction Module"
        style Shoreline fill:none,stroke:none
        Shoreline["SDS_shoreline: Segmentation & Polyline Generation"]:::process
    end

    subgraph "Transect Module"
        style Transect fill:none,stroke:none
        Transect["SDS_transects: Draw/Load Transects & Intersections"]:::process
    end

    subgraph "Tidal Correction & Slope"
        style Tidal fill:none,stroke:none
        Slope["SDS_slope: Tidal Correction & Slope Estimation"]:::process
        FES("FES2022 Tide Model"):::external
        ClipTide("examples/tide_model_clipping/clip_tide_files.py"):::ui
        FESDoc("doc/FES2022_setup.md"):::datastore
    end

    subgraph "Post-Processing & Analysis"
        style Post fill:none,stroke:none
        Post["SDS_tools: Despiking, Averages, Trend Estimation"]:::analysis
    end

    subgraph "Output & Visualization"
        style Output fill:none,stroke:none
        OutputNode["GeoJSON, .pkl, Plots, MP4 Timelapse"]:::analysis
    end

    %% Data Storage
    DataStore[("Local Filesystem\n(Raw, Intermediate, Outputs)")]:::datastore

    %% Data Flow
    UI1 -->|invokes| Download
    UI2 -->|invokes| Download
    UI3 -->|invokes| Download

    Download -->|calls API| EarthEngine
    Download -->|writes raw imagery| DataStore
    DataStore -->|reads raw imagery| Preprocess
    Preprocess -->|writes preprocessed imagery| DataStore
    DataStore -->|reads preprocessed imagery| Classify
    Models -->|loaded by| Classify
    Classify -->|writes classified raster| DataStore
    DataStore -->|reads classified raster| Shoreline
    Shoreline -->|writes shorelines| DataStore
    DataStore -->|reads shorelines| Transect
    Transect -->|requires tide data| Slope
    FES -->|provides tide model| Slope
    ClipTide -->|generates clipped tide files| DataStore
    FESDoc -.->|setup instructions| Slope
    Slope -->|writes corrected shorelines| DataStore
    DataStore -->|reads corrected shorelines| Post
    Post -->|calls| SDS_Tools[/"SDS_tools"/] 
    Post -->|writes final outputs| DataStore
    DataStore -->|serves outputs| OutputNode

    %% Click Events
    click UI1 "https://github.com/kvos/coastsat/blob/master/example.py"
    click UI2 "https://github.com/kvos/coastsat/blob/master/example_jupyter.ipynb"
    click UI3 "https://github.com/kvos/coastsat/blob/master/test.py"
    click Download "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_download.py"
    click Preprocess "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_preprocess.py"
    click Classify "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_classify.py"
    click Models "https://github.com/kvos/coastsat/blob/master/classification/models/*.pkl"
    click TrainNB "https://github.com/kvos/coastsat/blob/master/classification/train_new_classifier.ipynb"
    click Shoreline "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_shoreline.py"
    click Transect "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_transects.py"
    click Slope "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_slope.py"
    click FESDoc "https://github.com/kvos/coastsat/blob/master/doc/FES2022_setup.md"
    click ClipTide "https://github.com/kvos/coastsat/blob/master/examples/tide_model_clipping/clip_tide_files.py"
    click Post "https://github.com/kvos/coastsat/blob/master/coastsat/SDS_tools.py"

    %% Styles
    classDef ui fill:#FFF2CC,stroke:#333,stroke-width:1px,rx:5,ry:5
    classDef process fill:#FFE0B2,stroke:#333,stroke-width:1px
    classDef external fill:#BBDEFB,stroke:#333,stroke-width:1px,shape:cloud
    classDef datastore fill:#C8E6C9,stroke:#333,stroke-width:1px,shape:cylinder
    classDef analysis fill:#E1BEE7,stroke:#333,stroke-width:1px
```
