How shuyansy/earth-observation-vlms works
This repository contains two Earth-observation vision-language systems, EarthMind and TerraScope, with data handling, multimodal models, grounded segmentation, demos, and evaluation code. The principal workflow is a user providing Earth-observation imagery and a task, a model interpreting it and optionally producing grounded masks, and an answer or evaluation result. The README establishes the product family and its data, model, training, and evaluation scope; sampled source-index references support the internal model and dataset links shown. Other subsystem links are omitted where implementation wiring was not shown.
Main entry points: sam2_base.py Segmentation model · terrascope_model.py TerraScope model · sam2_utils.py SAM components · vqa_dataset.py Task datasets
Components 23
EarthMind
- EarthMind demo
EarthMind/demo.py - Task datasets
EarthMind/projects/llava_sam2/datasets/vqa_dataset.py - Batch collation
EarthMind/projects/glamm/datasets/collate_fns/glamm_collate_fn.py - Dataset utilities
EarthMind/projects/glamm/datasets/utils/utils.py - EarthMind model
EarthMind/projects/llava_sam2/models/llava_sam2.py - Segmentation model
EarthMind/third_parts/sam2/modeling/sam2_base.py - Video tracking
EarthMind/third_parts/sam2/sam2_video_predictor.py - Interactive app
EarthMind/projects/llava_sam2/gradio/app.py
TerraScope
- TerraScope demo
TerraScope/demo.py - Task datasets
TerraScope/projects/terrascope/datasets/GCG_Dataset.py - TerraScope model
TerraScope/projects/terrascope/models/terrascope_model.py - Vision-language core
TerraScope/projects/terrascope/models/internvl.py - Grounded reasoning
TerraScope/projects/terrascope/models/lisa.py - Image preprocessing
TerraScope/projects/terrascope/models/preprocess/image_resize.py - Interactive app
TerraScope/projects/terrascope/gradio/app.py
SAM2 segmentation
- Prompt encoder
EarthMind/third_parts/sam2/modeling/sam/prompt_encoder.py - Mask transformer
EarthMind/third_parts/sam2/modeling/sam/transformer.py - Mask decoder
EarthMind/third_parts/sam2/modeling/sam/mask_decoder.py - SAM components
EarthMind/third_parts/sam2/modeling/sam2_utils.py - Video tracking
TerraScope/third_parts/sam2/sam2_video_predictor.py
Evaluation
- EarthMind evaluation
EarthMind/projects/llava_sam2/evaluation/run_evaluations.py - Benchmark evaluation
TerraScope/projects/terrascope/evaluation/eval_terrascope_bench.py
Other components
- EO user
Connections 15
- EO user → EarthMind demo: provides task
- EO user → TerraScope demo: provides task
- Task datasets → Dataset utilities: uses
EarthMind/projects/llava_sam2/datasets/vqa_dataset.py - Task datasets → Batch collation: uses
EarthMind/projects/llava_sam2/datasets/vqa_dataset.py - TerraScope model → Vision-language core: uses
TerraScope/projects/terrascope/models/terrascope_model.py - TerraScope model → Grounded reasoning: uses
TerraScope/projects/terrascope/models/terrascope_model.py - TerraScope model → Image preprocessing: uses
TerraScope/projects/terrascope/models/terrascope_model.py - Segmentation model → Prompt encoder: uses
EarthMind/third_parts/sam2/modeling/sam2_base.py - Segmentation model → Mask transformer: uses
EarthMind/third_parts/sam2/modeling/sam2_base.py - Segmentation model → Mask decoder: uses
EarthMind/third_parts/sam2/modeling/sam2_base.py - Prompt encoder → SAM components: uses
EarthMind/third_parts/sam2/modeling/sam/prompt_encoder.py - Mask transformer → SAM components: uses
EarthMind/third_parts/sam2/modeling/sam/transformer.py - Mask decoder → SAM components: uses
EarthMind/third_parts/sam2/modeling/sam/mask_decoder.py - Video tracking → Segmentation model: uses
EarthMind/third_parts/sam2/sam2_video_predictor.py - Video tracking → Segmentation model: uses
TerraScope/third_parts/sam2/sam2_video_predictor.py