# barbarametzler/sat_embeddings architecture

> Architecture diagram of the GitHub repository barbarametzler/sat_embeddings, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/barbarametzler/sat_embeddings
- Repository: https://github.com/barbarametzler/sat_embeddings

## Overview

##### Project type and purpose
- This is a data-processing pipeline (an ETL-style tool) for extracting image embeddings from a large satellite composite using a pretrained transformer (Satlas) model.
- The outputs—parquet files of embeddings—are used for downstream ML tasks (regression, classification, unsupervised analysis).

##### File structure
- Top-level:
- README.md, environment.yml – documentation and environment spec
- code/ – main implementation
- code/ subfolders:
- create_tiles/ – scripts to slice the .vrt into .tif tiles (data preprocessing)
- create_embeddings/ – scripts to run Satlas and produce embeddings
- feature_space_analysis/ – notebooks for analyzing or visualizing the embedding space
- No web frontend or service layer—this is a batch pipeline.

##### README for additional insights
- Two main steps:
1) Data preprocessing: create_tiles.py takes a .vrt, H3 grid inputs, outputs folder of .tif tiles.
2) Create embeddings: create_embeddings_fpn_pool.py reads the tiles folder + model weights, outputs a parquet of embeddings.
- Dependencies: conda environment, PyTorch, optional CUDA, external data links.

##### Elements
- Blue for data sources (vrt, grid files)
- Green for processing modules (create_tiles, create_embeddings)
- Orange for outputs (tif folder, embeddings parquet)
- Purple for analysis layer (notebooks)

##### Detailed breakdown
1) Data Sources Layer (leftmost)
2) Preprocessing Layer
- Inputs: above data sources + size, mask flags
- Output: “Tiles Folder (.tif files)”
3) Embedding Layer
- Inputs: Tiles Folder, Satlas model weights (.pth)
- Output: “Embeddings.parquet”
4) Analysis & Usage Layer
5) Environment & Deployment Sidebar

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Sources Layer
    subgraph "Data Sources"
        DS1["Satellite Composite (.vrt)"]:::data
        DS2["H3 Grid Parquet"]:::data
        DS3["Grid420 Parquet"]:::data
    end

    %% Preprocessing Layer
    subgraph "Preprocessing"
        CT["create_tiles.py"]:::processing
        UT["tools_satellite_demoland.py"]:::processing
        TF["Tiles Folder (.tif)"]:::artifact
    end

    %% Embedding Generation Layer
    subgraph "Embedding Generation"
        EG["create_embeddings_fpn_pool.py"]:::processing
        MW["Satlas Model Weights (.pth)"]:::data
        EP["Embeddings.parquet"]:::artifact
    end

    %% Analysis & Downstream Layer
    subgraph "Analysis & Downstream"
        NB1["nearest_neighbour.ipynb"]:::analysis
        NB2["Demo_embeddings_clay_satlas.ipynb"]:::analysis
        ML["Downstream ML Tasks"]:::analysis
    end

    %% Environment & Documentation Sidebar
    subgraph "Environment & Documentation"
        ENV1["Conda env (environment.yml)"]:::env
        ENV2["PyTorch + CUDA"]:::env
        DOC["README.md"]:::env
    end

    %% Connections Data Sources → Preprocessing
    DS1 -->|"reads composite"| CT
    DS2 -->|"reads grid"| CT
    DS3 -->|"reads grid"| CT

    %% Preprocessing internals
    CT -->|"calls helper"| UT
    CT -->|"writes tiles"| TF

    %% Preprocessing → Embedding
    TF -->|"reads tiles"| EG
    MW -->|"loads weights"| EG
    EG -->|"writes embeddings"| EP

    %% Embedding → Analysis & Downstream
    EP -->|"inputs embeddings"| NB1
    EP -->|"inputs embeddings"| NB2
    EP -->|"inputs embeddings"| ML

    %% Environment & Documentation dependencies
    ENV1 --> CT
    ENV2 --> EG
    DOC --> DS1

    %% Click Events
    click CT "https://github.com/barbarametzler/sat_embeddings/blob/main/code/create_tiles/create_tiles.py"
    click UT "https://github.com/barbarametzler/sat_embeddings/blob/main/code/create_tiles/tools_satellite_demoland.py"
    click EG "https://github.com/barbarametzler/sat_embeddings/blob/main/code/create_embeddings/create_embeddings_fpn_pool.py"
    click NB1 "https://github.com/barbarametzler/sat_embeddings/blob/main/code/feature_space_analysis/nearest_neighbour.ipynb"
    click NB2 "https://github.com/barbarametzler/sat_embeddings/blob/main/code/Demo_embeddings_clay_satlas.ipynb"
    click ENV1 "https://github.com/barbarametzler/sat_embeddings/blob/main/environment.yml"
    click DOC "https://github.com/barbarametzler/sat_embeddings/blob/main/README.md"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef processing fill:#90EE90,stroke:#000,stroke-width:1px
    classDef artifact fill:#FFA500,stroke:#000,stroke-width:1px
    classDef analysis fill:#DDA0DD,stroke:#000,stroke-width:1px
    classDef env fill:#D3D3D3,stroke:#000,stroke-width:1px
```
