How barbarametzler/sat_embeddings works
Project type and purpose
• This is a data-processing pipeline (an ETL-style tool) for extracting image embeddings from a large satellite composite using a pretrained transformer (Satlas) model.
• The outputs—parquet files of embeddings—are used for downstream ML tasks (regression, classification, unsupervised analysis).
File structure
• Top-level:
• README.md, environment.yml – documentation and environment spec
• code/ – main implementation
• code/ subfolders:
• create_tiles/ – scripts to slice the .vrt into .tif tiles (data preprocessing)
• create_embeddings/ – scripts to run Satlas and produce embeddings
• feature_space_analysis/ – notebooks for analyzing or visualizing the embedding space
• No web frontend or service layer—this is a batch pipeline.
README for additional insights
• Two main steps:
1) Data preprocessing: create_tiles.py takes a .vrt, H3 grid inputs, outputs folder of .tif tiles.
2) Create embeddings: create_embeddings_fpn_pool.py reads the tiles folder + model weights, outputs a parquet of embeddings.
• Dependencies: conda environment, PyTorch, optional CUDA, external data links.
Elements
• Blue for data sources (vrt, grid files)
• Green for processing modules (create_tiles, create_embeddings)
• Orange for outputs (tif folder, embeddings parquet)
• Purple for analysis layer (notebooks)
Detailed breakdown
1) Data Sources Layer (leftmost)
2) Preprocessing Layer
• Inputs: above data sources + size, mask flags
• Output: “Tiles Folder (.tif files)”
3) Embedding Layer
• Inputs: Tiles Folder, Satlas model weights (.pth)
• Output: “Embeddings.parquet”
4) Analysis & Usage Layer
5) Environment & Deployment Sidebar