# visionxlab/crossearth architecture

> Architecture diagram of the GitHub repository visionxlab/crossearth, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/visionxlab/crossearth
- Repository: https://github.com/visionxlab/crossearth

## Overview

##### Project Type and Purpose
- CrossEarth is a research‐oriented machine learning library/toolkit rather than a typical front-end/ back-end web application.
- Its purpose is to provide a geospatial vision foundation model for domain‐generalizable semantic segmentation of remote sensing imagery. It bundles:
- Model code (backbones, heads, segmentors)
- Dataset handling (rs_dataset, tools/dataset_converters)
- Configuration management (configs/)
- Training/inference scripts (tools/)
- Model conversion utilities (convert_models/)
- Benchmarks and evaluation metrics

##### File Structure Analysis
- Top-level packages:
- CrossEarth/ – core model and metric code
– models/backbones, heads, segmentors
– dg_metrics.py
- rs_dataset/ – dataset definitions for remote sensing sources
- configs/ – hierarchical experiment configurations organized by dataset and model variant, built on _base_
- tools/ – CLI scripts for training (train.py), testing/inference (test.py), weight conversion, dataset conversion
- convert_models/ – specialized scripts to convert pretrained Vision Foundation Models (e.g., dinov2) to the CrossEarth format
- benchmarks/ – benchmark descriptions and evaluation protocols
- requirements.txt – dependency list (PyTorch, mmsegmentation, mmcv, mmdet, xformers, etc.)

##### Key Insights from README
- Leverages MMEngine, MMCV, MMSegmentation, MMDet, PyTorch 2.0, xFormers.
- Workflow: download pretrained weights → adjust config files (datasets + model settings) → python tools/test.py / tools/train.py.
- Domain generalization: many dataset cross‐domain experiments configured under configs/_base_/datasets.
- Outputs: segmentation masks, pseudo labels, checkpoints stored in CrossEarth/checkpoints.

a. Main Components
- Configuration Manager
- Reads experiment settings from configs/ (dataset paths, model parameters, training schedule).
- Data Pipeline
- rs_dataset module + tools/dataset_converters → dataset loader, augmentation, sampling.
- Model Module
- Backbones (e.g., dinov2, reins, vit variants)
- Heads (rein_mask2former)
- Segmentors (peft_encoder_decoder)
- Optimizer & Scheduler
- PEFT optimizer constructor in CrossEarth/optimizers + schedule_40k.py
- Training & Inference Engine
- tools/train.py orchestrates forward/backward passes, logging, checkpointing
- tools/test.py runs evaluation / pseudo label generation
- Conversion Utilities
- convert_models/*.py to adapt external pretrained weights
- Benchmarks & Metrics
- benchmarks/ + CrossEarth/dg_metrics.py for standardized evaluation

b. Component Interactions & Data Flow
1. User invokes CLI (train.py or test.py)
2. Config Manager loads experiment config → instantiates Dataset and Model
3. Data Pipeline reads raw imagery via rs_dataset → batches fed to Model
4. Model (Backbone → Head → Segmentor) outputs segmentation logits
5. In training:
- Optimizer updates model parameters
- Scheduler adjusts learning rate
- Checkpoints saved to disk
6. In inference:
- Model checkpoints loaded
- Predictions post-processed and saved as masks / pseudo labels
7. Benchmark scripts use dg_metrics.py to compute performance

c. Architectural Patterns & Design Principles
- Modular design: clear separation of dataset, model, optimizer, config, and CLI layers
- Plugin architecture (via MMEngine/MMCV): models and optimizers registered dynamically
- Configuration‐driven experiments: promotes reproducibility and easy domain-generalization experiments
- Conversion layer isolates weight adaptation from core training code

d. Key Technologies & Frameworks
- PyTorch 2.0 / CUDA 11.7
- OpenMIM / MMEngine / MMCV / MMSegmentation / MMDetection
- xFormers for efficient transformer backbones
- Hydra‐style / config‐file management in configs/

##### Diagramming Conventions
- Components as rectangles labeled clearly (e.g., “Backbone Module”, “Dataset Loader”)
- Blue for core modules (CrossEarth/)
- Green for data (rs_dataset, tools/dataset_converters)
- Orange for configs
- Purple for external services/dependencies

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Interface
    CLI["CLI (tools/train.py or tools/test.py)"]:::external

    %% Configuration Manager
    subgraph "Configuration Manager"
        Config["Configuration Manager"]:::config
    end

    %% Data Pipeline
    subgraph "Data Pipeline"
        DSDef["Dataset Definitions"]:::data
        Conv["Dataset Converters"]:::data
        DSLoader["Dataset Loader"]:::data
    end

    %% Model Module
    subgraph "Model Module"
        subgraph "Backbones"
            Backbone["Backbone Module"]:::core
        end
        subgraph "Heads"
            Head["Head Module"]:::core
        end
        subgraph "Segmentors"
            Segmentor["Segmentor Module"]:::core
        end
    end

    %% Optimizer & Scheduler
    subgraph "Optimizer & Scheduler"
        Opt["PEFT Optimizer Constructor"]:::core
        Schedule["LR Schedule"]:::config
    end

    %% Training & Inference Engine
    subgraph "Training & Inference Engine"
        Train["Training Engine"]:::core
        Test["Inference Engine"]:::core
    end

    %% Conversion Utilities
    subgraph "Conversion Utilities"
        Convert["Model Conversion Utilities"]:::data
    end

    %% Benchmarks & Metrics
    subgraph "Benchmarks & Metrics"
        Metrics["Domain‐Generalization Metrics"]:::core
        Bench["Benchmark Definitions"]:::core
    end

    %% External Dependencies
    subgraph "External Dependencies"
        Weights["Pretrained Weights"]:::external
        Libs["MMEngine/MMCV/MMSegmentation/MMDet/xFormers"]:::external
    end

    %% Data Flow & Interactions
    CLI --> Config
    Config --> DSDef
    Config --> Backbone
    Config --> Schedule
    DSDef --> DSLoader
    Conv --> DSLoader
    DSLoader --> Backbone
    Backbone --> Head
    Head --> Segmentor
    Segmentor --> Train
    Segmentor --> Test
    Train --> Opt
    Train --> Schedule
    Train --> CLI
    Test --> CLI
    Convert -.-> Backbone
    Convert -.-> Head
    Convert -.-> Segmentor
    Weights --> Convert
    Weights --> Train
    Libs --> Backbone
    Libs --> Head
    Libs --> Segmentor
    Train --> Bench
    Test --> Bench
    Bench --> Metrics
    Metrics --> Bench

    %% Click Events
    click Config "https://github.com/visionxlab/crossearth/tree/main/configs/"
    click DSDef "https://github.com/visionxlab/crossearth/tree/main/rs_dataset/"
    click Conv "https://github.com/visionxlab/crossearth/tree/main/tools/dataset_converters/"
    click Backbone "https://github.com/visionxlab/crossearth/tree/main/CrossEarth/models/backbones/"
    click Head "https://github.com/visionxlab/crossearth/tree/main/CrossEarth/models/heads/"
    click Segmentor "https://github.com/visionxlab/crossearth/tree/main/CrossEarth/models/segmentors/"
    click Metrics "https://github.com/visionxlab/crossearth/blob/main/CrossEarth/dg_metrics.py"
    click Opt "https://github.com/visionxlab/crossearth/blob/main/CrossEarth/optimizers/peft_optimizer_constructor.py"
    click Schedule "https://github.com/visionxlab/crossearth/blob/main/configs/_base_/schedules/schedule_40k.py"
    click Train "https://github.com/visionxlab/crossearth/blob/main/tools/train.py"
    click Test "https://github.com/visionxlab/crossearth/blob/main/tools/test.py"
    click Convert "https://github.com/visionxlab/crossearth/tree/main/convert_models/"
    click Bench "https://github.com/visionxlab/crossearth/tree/main/benchmarks/"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef data fill:#90EE90,stroke:#333,stroke-width:1px
    classDef config fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#DDA0DD,stroke:#333,stroke-width:1px
```
