# aws-samples/sample-geospatial-foundation-models-on-aws architecture

> Architecture diagram of the GitHub repository aws-samples/sample-geospatial-foundation-models-on-aws, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aws-samples/sample-geospatial-foundation-models-on-aws
- Repository: https://github.com/aws-samples/sample-geospatial-foundation-models-on-aws

## Overview

##### Project Type and Purpose
- This is a full‐stack, cloud‐native web application demonstrating Geospatial Foundation Models (GeoFMs) for ecosystem monitoring.
- It combines a Solara‐based frontend, serverless backend Lambdas (API surface for tile serving, similarity search, and change detection), asynchronous SageMaker pipelines (for embedding generation and model fine‐tuning), and several AWS managed services (ElastiCache, OpenSearch, S3, Cognito, Fargate, CloudFront).

##### File Structure Analysis
- Top‐level
- `sagemaker_pipelines/` – Jupyter notebooks and Python scripts to build Docker images, instantiate SageMaker pipelines for embedding generation (prerequisites & embedding_generation).
- `ui/geofm-demo-stack/` – AWS CDK stack (TypeScript) with subfolders:
◦ `bin/` and `lib/` – CDK entrypoints and stack definitions (auth, backend, frontend stacks).
◦ `lambdas/` – three Lambda functions: API‐auth, tile server (TiTiler), Cognito signup, similarity search, change detection.
◦ `solara-fe/` – Python Solara app, Dockerfile, UI pages for home, similarity search, change detection.
◦ `ui-assets/` – index.html, CSS.
◦ CDK config files (`cdk.json`, `tsconfig.json`).
- Presence of Dockerfiles under prerequisites and `solara-fe` indicates containerized compute for SageMaker jobs and Fargate.
- CDK test folder shows unit tests for stack.

##### README Insights
- Three-tier architecture: Frontend (Solara on Fargate via CloudFront + Cognito), AI/ML & Analytics Tier (SageMaker pipelines + Lambdas), Data Tier (S3 for imagery, ElastiCache for tile caching, OpenSearch & LanceDB for vector search).
- Two asynchronous SageMaker pipelines: embedding generation (preprocessing + model inference + load to LanceDB) and fine‐tuning (not in file tree but implied).

##### Main Components
a. Frontend Tier
- CloudFront CDN → Fargate container running Solara FE
- Amazon Cognito User Pool for authentication
b. Backend / AI & Analytics Tier
- API Gateway or direct Lambda endpoints:
- Tile server Lambda (TiTiler) → ElastiCache (tile cache) → S3 Sentinel‐2 imagery
- Search Lambda → OpenSearch Service & LanceDB vector DB
- Change‐detection Lambda → OpenSearch time‐series queries
- SageMaker Pipelines:
- Prerequisite Docker builds (Clay GPU image, geo‐processing image) → ECR
- Embedding Generation Pipeline (S3 data fetch, preprocessing, inference with GeoFM from HuggingFace, output to LanceDB & S3 config.json)
- (Optionally) Fine‐tuning Pipeline
c. Data Tier
- S3 buckets (raw imagery, pipeline outputs, config files)
- Amazon ElastiCache Redis (tile caching)
- Amazon OpenSearch Service (vector index & time‐series data)
- LanceDB (vector DB, potentially on SageMaker or VPC)

##### Relationships and Data Flow
- User → CloudFront → Solara FE (via HTTPS, authenticated by Cognito)
- Solara FE → API endpoints (tile server, search, change‐detect)
- Tile server Lambda: reads from S3 / Redis cache, returns map tiles
- Search Lambda: queries OpenSearch & LanceDB, returns nearest embeddings
- Change‐detect Lambda: fetches historical embeddings from OpenSearch, computes anomalies
- SageMaker Pipeline jobs: triggered manually or via SDK, read from S3, run on managed instances with custom Docker images in ECR, write embeddings to LanceDB & metadata to S3

##### Architectural Patterns & Design Principles
- Three‐tier separation (frontend, compute, data)
- Serverless microservices (AWS Lambda, API Gateway)
- Asynchronous batch processing (SageMaker Pipelines)
- Infrastructure‐as‐Code (CDK) for reproducibility
- Containerization for portability (Docker images for SageMaker & Fargate)
- Security & auth via Cognito

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Tier
    subgraph "UI Tier" 
        direction TB
        CF["CloudFront CDN"]:::ui
        Cognito["Cognito User Pool"]:::ui
        FE["Solara Frontend (Fargate)"]:::ui
    end

    %% Compute Tier
    subgraph "Compute Tier"
        direction TB
        API["API Gateway"]:::compute
        subgraph "Lambda Functions"
            direction TB
            tile["Tile Server Lambda (TiTiler)"]:::compute
            search["Search Lambda"]:::compute
            change["Change-Detection Lambda"]:::compute
        end
    end

    %% ML Pipelines
    subgraph "ML Pipelines"
        direction TB
        subgraph "Prerequisite Builds"
            direction TB
            clay["Clay GPU Dockerfile"]:::ml
            geoimg["Geospatial Processing Dockerfile"]:::ml
            ECR["ECR (Docker Images)"]:::ml
        end
        subgraph "Embedding Generation Pipeline"
            direction TB
            pipeline["Embedding Generation Pipeline"]:::ml
        end
    end

    %% Data Tier
    subgraph "Data Tier"
        direction TB
        S3["Amazon S3"]:::data
        Redis["ElastiCache Redis"]:::data
        OS["OpenSearch Service"]:::data
        Lance["LanceDB Vector DB"]:::data
    end

    %% User Flow
    User["User"]:::external
    User -->|"HTTPS"| CF
    CF -->|"Authenticated via Cognito"| FE
    Cognito --> FE
    FE -->|"REST API calls"| API
    API -->|"tile request"| tile
    API -->|"vector search"| search
    API -->|"time-series query"| change

    %% Lambda Data Access
    tile -->|"Redis GET"| Redis
    Redis -->|"cache miss? S3 GET"| S3
    search -->|"search query"| OS
    search -->|"vector lookup"| Lance
    change -->|"historical data query"| OS

    %% ML Pipeline Flow
    clay --> ECR
    geoimg --> ECR
    ECR --> pipeline
    S3 --> pipeline
    pipeline -->|"write embeddings"| Lance
    pipeline -->|"write config"| S3

    %% Click Events
    click clay "https://github.com/aws-samples/sample-geospatial-foundation-models-on-aws/tree/main/sagemaker_pipelines/prerequisites/clay_gpu_docker_image/Dockerfile"
    click geoimg "https://github.com/aws-samples/sample-geospatial-foundation-models-on-aws/tree/main/sagemaker_pipelines/prerequisites/geospatial_processing_image/Dockerfile"
    click pipeline "https://github.com/aws-samples/sample-geospatial-foundation-models-on-aws/blob/main/sagemaker_pipelines/embedding_generation/embedding_generation_pipeline.ipynb"
    click FE "https://github.com/aws-samples/sample-geospatial-foundation-models-on-aws/blob/main/ui/geofm-demo-stack/solara-fe/app.py"

    %% Styles
    classDef ui fill:#E3F2FD,stroke:#90CAF9,color:#0D47A1
    classDef compute fill:#E8F5E9,stroke:#66BB6A,color:#1B5E20
    classDef data fill:#FFF3E0,stroke:#FFB74D,color:#E65100
    classDef ml fill:#F3E5F5,stroke:#CE93D8,color:#4A148C
    classDef external fill:#ECEFF1,stroke:#B0BEC5,color:#37474F
    class User external
    class CF,Cognito,FE ui
    class API,tile,search,change compute
    class S3,Redis,OS,Lance data
    class clay,geoimg,pipeline,ECR ml
```
