# ngageoint/sarpy architecture

> Architecture diagram of the GitHub repository ngageoint/sarpy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ngageoint/sarpy
- Repository: https://github.com/ngageoint/sarpy

## Overview

##### Project Type and Purpose
- This is a Python library (not a web or desktop app) for reading, writing, and performing simple processing on Synthetic Aperture Radar (SAR) data in a variety of standardized formats (SICD, SIDD, CPHD, CRSD) plus vendor-specific single look complex (SLC) files.
- It is intended as a building block for downstream applications, complementary to C++ and MATLAB toolkits.

##### High-Level Architecture
The library follows a layered, modular design with clear separation of concerns:
A. Core Library (sarpy/)
– annotation: classes and schemas for metadata annotation
– geometry: coordinate transforms, geocoords, projections
– io: file readers and writers organized by format
- DEM, complex, general (TIFF/NITF base), phase_history, product, received, xml
- Each submodule has base classes, converters, schema definitions
– processing: algorithms (ortho-rectify, registration, SICD normalization, SIDD product creation)
– consistency: metadata validation and consistency checks
– utils: helper functions for conversions, KMZ, NITF utilities, noise models
– visualization: KMZ generation, remapping overlays
B. External Dependencies
– Required: numpy, scipy
– Optional (runtime‐loaded): h5py, pillow, lxml, networkx, shapely, pyproj, sphinx, sphinx_gallery
C. Testing & Examples
– tests/: unit tests organized by module
– examples/: Python scripts demonstrating key workflows
D. Build & CI
– setup.py / setup.cfg for packaging
– GitHub Actions workflow (pypi.yml) to run tests and publish releases
– Sphinx docs (docs/*), ReadTheDocs integration

##### Main Components
- Users / Client Code: imports sarpy, calls high-level API
- File System / Data Sources: SICD, SIDD, CPHD, CRSD, vendor SLC files, GeoTIFF DEM, NITF images
- IO Layer: format detectors → specific Reader classes → metadata + image data objects
- Core Models: in-memory Python objects (annotation, geometry, base image data)
- Processing Layer: algorithms consuming core models, producing derived models or processed data
- Utility Layer: cross-cutting helpers (unit conversions, KMZ packaging)
- Visualization Layer: KMZ & map creation modules
- External Libraries: numpy/scipy (core), optional libs invoked by IO & processing modules
- Testing Harness & CI: test runner, GitHub Actions
- Documentation Generator: Sphinx build subsystem

##### Relationships and Data Flow
1. Client → (reads) → IO Layer
2. IO Layer → (instantiates) → Core Models
3. Core Models → (fed into) → Processing Layer
4. Processing Layer → (update/produce) → Core Models or output files
5. Core Models → (utilities) → Visualization / KMZ / helpers
6. CI Pipeline → (executes) → tests / doc build / package publish

##### Architectural Patterns & Principles
- Layered Architecture: IO → Core Model → Processing → Utilities → Visualization
- Plugin-style optional dependencies: runtime import with fallbacks
- Package modularity: each subpackage encapsulates domain area
- Schema-driven design: XSD schemas and element classes mirror SAR standards
- Separation of data representation (annotation, geometry) from processing logic

##### Technologies & Frameworks
- Python packaging: setuptools (setup.py/ cfg), PyPI
- Testing: pytest, nox (noxfile.py)
- Documentation: Sphinx (docs/), ReadTheDocs config
- CI/CD: GitHub Actions (.github/workflows/pypi.yml)
- Scientific stack: numpy, scipy
- Optional GIS/spatial: h5py, pillow, lxml, networkx, shapely, pyproj

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Users
    User["Client Code"]:::lib

    %% Data Sources
    subgraph "Data Sources"
        FS_SICD(("SICD")):::dataSource
        FS_SIDD(("SIDD")):::dataSource
        FS_CPhD(("CPHD")):::dataSource
        FS_CRSD(("CRSD")):::dataSource
        FS_SLC(("Vendor SLC")):::dataSource
        FS_DEM(("GeoTIFF DEM")):::dataSource
        FS_NITF(("NITF")):::dataSource
    end

    %% IO Layer
    subgraph "sarpy.io" 
        ioBase["__init__.py"]:::lib
        subgraph "DEM"
            io_DEM_DEM["DEM.py"]:::lib
            io_DEM_DT["DTED.py"]:::lib
        end
        subgraph "complex"
            io_C_base["base.py"]:::lib
            io_C_sicd["sicd.py"]:::lib
        end
        subgraph "general"
            io_G_base["base.py"]:::lib
            io_G_nitf["nitf.py"]:::lib
        end
        io_Phase["cphd.py"]:::lib
        io_Product["sidd.py"]:::lib
        io_Received["crsd.py"]:::lib
        io_XML["descriptors.py"]:::lib
    end

    %% Core Modules
    subgraph "sarpy.annotation"
        Annotation["__init__.py"]:::lib
    end
    subgraph "sarpy.geometry"
        Geometry["__init__.py"]:::lib
    end
    subgraph "Core Models"
        subgraph "annotation elements"
            Core_AnnElem["CollectionInfo.py"]:::lib
        end
        subgraph "geometry elements"
            Core_GeomElem["geometry_elements.py"]:::lib
        end
    end

    %% Processing Layer
    subgraph "sarpy.processing"
        Proc_Ortho["ortho_methods.py"]:::lib
        Proc_Reg["regi.py"]:::lib
        Proc_SICD["normalize_sicd.py"]:::lib
        Proc_SIDD["sidd_product_creation.py"]:::lib
    end

    Consistency["consistency.py"]:::lib

    %% Utilities
    Util_KMZ["create_kmz.py"]:::lib

    %% Visualization
    Viz_KMZ["kmz_product_creation.py"]:::lib
    Viz_Remap["remap.py"]:::lib

    %% External Dependencies
    subgraph "External Dependencies"
        Numpy(("numpy")):::ext
        Scipy(("scipy")):::ext
        OptionalLibs(("h5py, pillow, lxml, networkx, shapely, pyproj")):::ext
    end

    %% Testing & CI
    subgraph "Testing & CI"
        Tests(("tests/")):::ci
        Examples(("examples/first_example.py")):::ci
        CI[".github/workflows/pypi.yml"]:::ci
        Docs(("docs/")):::ci
        Package(("setup.py / setup.cfg")):::ci
    end

    %% Data Flow
    User --> ioBase
    FS_SICD --> io_C_sicd
    FS_SIDD --> io_Product
    FS_CPhD --> io_Phase
    FS_CRSD --> io_Received
    FS_SLC --> io_C_base
    FS_DEM --> io_DEM_DEM
    FS_NITF --> io_G_nitf

    ioBase --> io_DEM_DEM & io_DEM_DT & io_C_base & io_C_sicd & io_G_base & io_G_nitf & io_Phase & io_Product & io_Received & io_XML

    io_DEM_DEM --> Core_AnnElem & Core_GeomElem
    io_DEM_DT --> Core_AnnElem & Core_GeomElem
    io_C_base --> Core_AnnElem & Core_GeomElem
    io_C_sicd --> Core_AnnElem & Core_GeomElem
    io_G_base --> Core_AnnElem & Core_GeomElem
    io_G_nitf --> Core_AnnElem & Core_GeomElem
    io_Phase --> Core_AnnElem & Core_GeomElem
    io_Product --> Core_AnnElem & Core_GeomElem
    io_Received --> Core_AnnElem & Core_GeomElem
    io_XML --> Core_AnnElem & Core_GeomElem

    Core_AnnElem --> Proc_Ortho & Proc_Reg & Proc_SICD & Proc_SIDD
    Core_GeomElem --> Proc_Ortho & Proc_Reg & Proc_SICD & Proc_SIDD

    Proc_Ortho --> Consistency
    Proc_Reg --> Consistency
    Proc_SICD --> Consistency
    Proc_SIDD --> Consistency

    Consistency --> Util_KMZ
    Util_KMZ --> Viz_KMZ & Viz_Remap

    %% Dependencies usage
    Numpy --> ioBase & Proc_Ortho & Proc_Reg & Proc_SICD & Proc_SIDD
    Scipy --> ioBase & Proc_Ortho & Proc_Reg & Proc_SICD & Proc_SIDD
    OptionalLibs --> io_XML & Proc_SIDD

    %% CI Pipeline
    CI --> Tests
    CI --> Docs
    CI --> Package

    %% Click Events
    click Annotation "https://github.com/ngageoint/sarpy/blob/master/sarpy/annotation/__init__.py"
    click Geometry "https://github.com/ngageoint/sarpy/blob/master/sarpy/geometry/__init__.py"
    click ioBase "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/__init__.py"
    click io_DEM_DEM "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/DEM/DEM.py"
    click io_DEM_DT "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/DEM/DTED.py"
    click io_C_base "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/complex/base.py"
    click io_C_sicd "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/complex/sicd.py"
    click io_G_base "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/general/base.py"
    click io_G_nitf "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/general/nitf.py"
    click io_Phase "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/phase_history/cphd.py"
    click io_Product "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/product/sidd.py"
    click io_Received "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/received/crsd.py"
    click io_XML "https://github.com/ngageoint/sarpy/blob/master/sarpy/io/xml/descriptors.py"
    click Core_AnnElem "https://github.com/ngageoint/sarpy/blob/master/sarpy/annotation/afrl_rde_elements/CollectionInfo.py"
    click Core_GeomElem "https://github.com/ngageoint/sarpy/blob/master/sarpy/geometry/geometry_elements.py"
    click Proc_Ortho "https://github.com/ngageoint/sarpy/blob/master/sarpy/processing/ortho_rectify/ortho_methods.py"
    click Proc_Reg "https://github.com/ngageoint/sarpy/blob/master/sarpy/processing/registration/regi.py"
    click Proc_SICD "https://github.com/ngageoint/sarpy/blob/master/sarpy/processing/sicd/normalize_sicd.py"
    click Proc_SIDD "https://github.com/ngageoint/sarpy/blob/master/sarpy/processing/sidd/sidd_product_creation.py"
    click Consistency "https://github.com/ngageoint/sarpy/blob/master/sarpy/consistency/consistency.py"
    click Util_KMZ "https://github.com/ngageoint/sarpy/blob/master/sarpy/utils/create_kmz.py"
    click Viz_KMZ "https://github.com/ngageoint/sarpy/blob/master/sarpy/visualization/kmz_product_creation.py"
    click Viz_Remap "https://github.com/ngageoint/sarpy/blob/master/sarpy/visualization/remap.py"
    click Tests "https://github.com/ngageoint/sarpy/tree/master/tests/"
    click Examples "https://github.com/ngageoint/sarpy/blob/master/examples/first_example.py"
    click CI "https://github.com/ngageoint/sarpy/blob/master/.github/workflows/pypi.yml"
    click Docs "https://github.com/ngageoint/sarpy/tree/master/docs/"
    click Package "https://github.com/ngageoint/sarpy/blob/master/setup.py"

    %% Styles
    classDef lib fill:#ddebf7,stroke:#1f497d;
    classDef ext fill:#e2efda,stroke:#548235;
    classDef dataSource fill:#fce4d6,stroke:#c65911;
    classDef ci fill:#f4cccc,stroke:#a61c00;
```
