# ec-jrc/nrt architecture

> Architecture diagram of the GitHub repository ec-jrc/nrt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ec-jrc/nrt
- Repository: https://github.com/ec-jrc/nrt

## Overview

1. **What this repository is**
   - `nrt` is a **Python library**, not an application service.
   - Its core purpose is **near-real-time change detection** in **spatio-temporal satellite image time series**.
   - The architectural center is a **stateful monitoring workflow**: fit a model on a stable history period, then process one new acquisition at a time and flag breaks/anomalies.

2. **Primary data model and boundary**
   - The package is built around **array-based geospatial data**, especially **xarray DataArray/Dataset** objects and NumPy arrays.
   - Typical inputs are satellite cubes, masks, and time slices; typical outputs are break detections, monitoring state, reports, and serialized model state.
   - This library boundary is important: it does not manage data ingestion pipelines itself, but expects already prepared time-series data and masks.

3. **Main runtime flow**
   - The standard flow is:
     1. select a **stable history period**
     2. **fit** a model to the expected normal behavior
     3. enter **monitoring mode**
     4. ingest each new observation sequentially
     5. compare observation vs predicted behavior
     6. emit a **break / disturbance signal** when divergence persists
   - The API is designed for **online / incremental monitoring**, which makes the monitor object stateful between acquisitions.

4. **Core algorithm subsystem**
   - The `nrt.monitor` package contains the algorithm implementations:
     - `ewma.py` — Exponentially Weighted Moving Average
     - `cusum.py` — Cumulative Sum of residuals
     - `mosum.py` — Moving Sum of residuals
     - `ccdc.py` — Continuous Change Detection and Classification of land cover
     - `iqr.py` — interquartile-range based outlier detection
   - These algorithms share the same conceptual pipeline but differ in how residuals are accumulated, thresholds are computed, and breaks are confirmed.
   - `CCDC` is explicitly described as a **partial implementation** of the published method.

5. **Model fitting layer**
   - `nrt/fit_methods.py` provides the fitting routines used to build the “normal behavior” model.
   - The README and example show a **linear/harmonic trend model** being fit on the history period before monitoring begins.
   - This layer is the bridge between raw time series and monitoring algorithms: it defines the expected signal against which future observations are compared.

6. **Shared analysis utilities**
   - `stats.py`, `outliers.py`, `utils.py`, and `utils_efp.py` contain shared numerical and helper logic used across monitors.
   - These modules act as a common support layer for:
     - statistical calculations
     - residual handling
     - thresholding / outlier decisions
     - event or breakpoint-related utilities
   - `log.py` provides logging support for operational use.

7. **State, persistence, and operational continuity**
   - The monitors are designed to retain state across time steps, which matters in near-real-time operations.
   - The README calls out two key operational outputs:
     - `report(...)` for generating a monitoring report
     - `to_netcdf(...)` for saving monitor state to disk
   - NetCDF support is a meaningful architecture point because it allows stop/restart of long-running monitoring jobs and interoperates with geospatial data tooling.

8. **Technology stack that shapes the design**
   - The package depends on:
     - **NumPy** and **SciPy** for numeric/statistical computation
     - **xarray** for labeled multi-dimensional data
     - **numba** for performance on compute-heavy routines
     - **rasterio** and **netCDF4** for geospatial/raster and persistent scientific data formats
   - This stack indicates the package is optimized for **scientific array processing at scale**, rather than general-purpose application logic.

9. **Examples, docs, and distribution boundary**
   - The `docs/` tree and gallery scripts demonstrate intended usage patterns, including parallel computing and simulation data.
   - The package is documented with Read the Docs and distributed as a standard Python package via `pyproject.toml`.
   - The architecture therefore includes a clear separation between **library code**, **example workflows**, and **published API documentation**.

10. **Testing and validation structure**
   - `tests/unit_tests` validates the numerical helpers and fitting logic against fixture CSV data.
   - `tests/integration_tests` exercises the monitor implementations end-to-end.
   - This split reflects the architecture: lower-level math utilities are tested independently, while monitor classes are validated as operational pipelines.

11. **What to draw in an architecture diagram**
   - A useful diagram would show:
     - **Satellite time-series inputs** and masks feeding into
     - a **history fitting stage**
     - then into one of several **monitoring algorithm modules**
     - with shared **stats/utilities** underneath
     - and **state/report persistence** via NetCDF
   - The most important boundary is that the system is a **single-process analytical library with pluggable monitoring strategies**, not a distributed service.

Main entry points: [`nrt/monitor/__init__.py`](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/__init__.py) (Monitor API), [`nrt/stats.py`](https://github.com/ec-jrc/nrt/blob/master/nrt/stats.py) (Stats), [`nrt/fit_methods.py`](https://github.com/ec-jrc/nrt/blob/master/nrt/fit_methods.py) (Fit models), [`tests/unit_tests`](https://github.com/ec-jrc/nrt/tree/master/tests/unit_tests) (Unit tests)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core library"]
  node_inputs[("Satellite series<br/>geospatial inputs")]
  node_arrays["xarray/NumPy<br/>data model"]
  node_fit_methods["Fit models<br/>fitting layer<br/>[fit_methods.py]"]
  node_monitor_core["Monitor API<br/>stateful orchestration<br/>[__init__.py]"]
  node_ewma["EWMA<br/>change detector<br/>[ewma.py]"]
  node_cusum["CUSUM<br/>change detector<br/>[cusum.py]"]
  node_mosum["MOSUM<br/>change detector<br/>[mosum.py]"]
  node_ccdc["CCDC<br/>change detector<br/>[ccdc.py]"]
  node_iqr["IQR<br/>outlier detector<br/>[iqr.py]"]
  node_stats["Stats<br/>numerical support<br/>[stats.py]"]
  node_outliers["Outliers<br/>decision support<br/>[outliers.py]"]
  node_utils["Utils<br/>helpers<br/>[utils.py]"]
  node_utils_efp["EFP utils<br/>event utilities<br/>[utils_efp.py]"]
  node_log["Logging<br/>ops support<br/>[log.py]"]
  node_state_io[("State I/O<br/>persistence")]
end

subgraph group_docs["Docs & examples"]
  node_docs_gallery["Docs & gallery<br/>documentation"]
end

subgraph group_tests["Tests"]
  node_unit_tests["Unit tests<br/>validation"]
  node_integration_tests["Integration tests<br/>validation"]
end

node_inputs -->|"stored as"| node_arrays
node_arrays -->|"fit history"| node_fit_methods
node_fit_methods -->|"initialize"| node_monitor_core
node_monitor_core -->|"strategy"| node_ewma
node_monitor_core -->|"strategy"| node_cusum
node_monitor_core -->|"strategy"| node_mosum
node_monitor_core -->|"strategy"| node_ccdc
node_monitor_core -->|"strategy"| node_iqr
node_ewma -->|"uses"| node_stats
node_cusum -->|"uses"| node_stats
node_mosum -->|"uses"| node_stats
node_ccdc -->|"uses"| node_stats
node_iqr -->|"uses"| node_outliers
node_stats -->|"supports"| node_utils
node_outliers -->|"supports"| node_utils
node_utils_efp -->|"supports"| node_stats
node_monitor_core -->|"persist/report"| node_state_io
node_state_io -->|"logs"| node_log
node_docs_gallery -.->|"demonstrates"| node_fit_methods
node_docs_gallery -.->|"demonstrates"| node_monitor_core
node_unit_tests -->|"verifies"| node_fit_methods
node_unit_tests -->|"verifies"| node_stats
node_unit_tests -->|"verifies"| node_outliers
node_integration_tests -->|"verifies"| node_monitor_core
node_integration_tests -->|"verifies"| node_ccdc

click node_fit_methods "https://github.com/ec-jrc/nrt/blob/master/nrt/fit_methods.py"
click node_monitor_core "https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/__init__.py"
click node_ewma "https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/ewma.py"
click node_cusum "https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/cusum.py"
click node_mosum "https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/mosum.py"
click node_ccdc "https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/ccdc.py"
click node_iqr "https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/iqr.py"
click node_stats "https://github.com/ec-jrc/nrt/blob/master/nrt/stats.py"
click node_outliers "https://github.com/ec-jrc/nrt/blob/master/nrt/outliers.py"
click node_utils "https://github.com/ec-jrc/nrt/blob/master/nrt/utils.py"
click node_utils_efp "https://github.com/ec-jrc/nrt/blob/master/nrt/utils_efp.py"
click node_log "https://github.com/ec-jrc/nrt/blob/master/nrt/log.py"
click node_docs_gallery "https://github.com/ec-jrc/nrt/tree/master/docs"
click node_unit_tests "https://github.com/ec-jrc/nrt/tree/master/tests/unit_tests"
click node_integration_tests "https://github.com/ec-jrc/nrt/tree/master/tests/integration_tests"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_inputs,node_arrays,node_fit_methods,node_monitor_core,node_ewma,node_cusum,node_mosum,node_ccdc,node_iqr,node_stats,node_outliers,node_utils,node_utils_efp,node_log,node_state_io toneBlue
class node_docs_gallery toneAmber
class node_unit_tests,node_integration_tests toneMint
```

## Components

### Core library

Library boundary for fitting, monitoring, and shared numerical helpers.

- **Satellite series**: geospatial inputs · Prepared time-series cubes and masks used as monitor inputs.
- **xarray/NumPy**: data model · Labeled arrays and ndarrays used throughout the library.
- [Fit models](https://github.com/ec-jrc/nrt/blob/master/nrt/fit_methods.py): fitting layer · Builds the expected normal-behavior model from stable history. · `nrt/fit_methods.py`
- [Monitor API](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/__init__.py): stateful orchestration · Shared monitor interface and online processing flow. · `nrt/monitor/__init__.py`
- [EWMA](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/ewma.py): change detector · Exponentially weighted moving average monitor. · `nrt/monitor/ewma.py`
- [CUSUM](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/cusum.py): change detector · Cumulative-sum residual monitor. · `nrt/monitor/cusum.py`
- [MOSUM](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/mosum.py): change detector · Moving-sum residual monitor. · `nrt/monitor/mosum.py`
- [CCDC](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/ccdc.py): change detector · Partial implementation of continuous change detection and classification. · `nrt/monitor/ccdc.py`
- [IQR](https://github.com/ec-jrc/nrt/blob/master/nrt/monitor/iqr.py): outlier detector · Interquartile-range based anomaly detection. · `nrt/monitor/iqr.py`
- [Stats](https://github.com/ec-jrc/nrt/blob/master/nrt/stats.py): numerical support · Shared statistical calculations used by fitting and monitors. · `nrt/stats.py`
- [Outliers](https://github.com/ec-jrc/nrt/blob/master/nrt/outliers.py): decision support · Residual and threshold logic for anomaly decisions. · `nrt/outliers.py`
- [Utils](https://github.com/ec-jrc/nrt/blob/master/nrt/utils.py): helpers · Common helper routines shared across the package. · `nrt/utils.py`
- [EFP utils](https://github.com/ec-jrc/nrt/blob/master/nrt/utils_efp.py): event utilities · Breakpoint and event-related utilities. · `nrt/utils_efp.py`
- [Logging](https://github.com/ec-jrc/nrt/blob/master/nrt/log.py): ops support · Operational logging support for library use. · `nrt/log.py`
- **State I/O**: persistence · Report generation and NetCDF serialization for pause/resume monitoring.

### Docs & examples

User-facing documentation and gallery workflows.

- [Docs & gallery](https://github.com/ec-jrc/nrt/tree/master/docs): documentation · Reference docs and example workflows, including parallel computing and simulation demos. · `docs`

### Tests

Unit and integration validation of math helpers and monitors.

- [Unit tests](https://github.com/ec-jrc/nrt/tree/master/tests/unit_tests): validation · Tests for fitting logic and shared numerical helpers. · `tests/unit_tests`
- [Integration tests](https://github.com/ec-jrc/nrt/tree/master/tests/integration_tests): validation · End-to-end monitor tests for operational behavior. · `tests/integration_tests`

## Connections

- Satellite series → xarray/NumPy: stored as · Inputs are handled as labeled arrays and ndarrays.
- xarray/NumPy → Fit models: fit history · History-period arrays are used to fit the expected signal.
- Fit models → Monitor API: initialize · Fitted model state is passed into online monitoring.
- Monitor API → EWMA: strategy · EWMA is one monitoring implementation.
- Monitor API → CUSUM: strategy · CUSUM is one monitoring implementation.
- Monitor API → MOSUM: strategy · MOSUM is one monitoring implementation.
- Monitor API → CCDC: strategy · CCDC is one monitoring implementation.
- Monitor API → IQR: strategy · IQR is one monitoring implementation.
- EWMA → Stats: uses · Algorithms depend on shared statistics.
- CUSUM → Stats: uses · Algorithms depend on shared statistics.
- MOSUM → Stats: uses · Algorithms depend on shared statistics.
- CCDC → Stats: uses · Algorithms depend on shared statistics.
- IQR → Outliers: uses · Outlier decisions reuse shared threshold logic.
- Stats → Utils: supports · Shared helpers underpin numerical routines.
- Outliers → Utils: supports · Shared helpers underpin decision logic.
- EFP utils → Stats: supports · Breakpoint utilities feed statistical routines.
- Monitor API → State I/O: persist/report · Monitoring state can be reported and saved to NetCDF.
- State I/O → Logging: logs · Operational persistence and reporting are logged.
- Docs & gallery → Fit models: demonstrates · Documentation shows fitting workflows.
- Docs & gallery → Monitor API: demonstrates · Documentation shows monitoring workflows.
- Unit tests → Fit models: verifies · Unit tests cover fitting routines.
- Unit tests → Stats: verifies · Unit tests cover numerical helpers.
- Unit tests → Outliers: verifies · Unit tests cover outlier logic.
- Integration tests → Monitor API: verifies · Integration tests exercise end-to-end monitors.
- Integration tests → CCDC: verifies · CCDC has dedicated integration coverage.
