# japq91/smn_toe architecture

> Architecture diagram of the GitHub repository japq91/smn_toe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/japq91/smn_toe
- Repository: https://github.com/japq91/smn_toe

## Overview

smn_toe is a climate-research workflow analyzing CMIP6 and ERSSTv5 sea-surface temperatures (SST) and ENSO in Niño 1+2 and Niño 3.4. The documented E1 workflow obtains and prepares data; E2 evaluates historical model skill and selects models; E3 analyzes future projections and warm events. E4 uncertainty partitioning and Time of Emergence (TOE) are documented as pending, so they are not shown as implemented. Connections use README workflow statements and shown source references; some documented processing steps have no matching file in the supplied tree.

Main entry points: [`scripts/01_query_esgf_catalog.py`](https://github.com/japq91/smn_toe/blob/main/scripts/01_query_esgf_catalog.py) (ESGF catalog), [`scripts/05_apply_ocean_mask.py`](https://github.com/japq91/smn_toe/blob/main/scripts/05_apply_ocean_mask.py) (Ocean masking), [`scripts_e3/common_e3.py`](https://github.com/japq91/smn_toe/blob/main/scripts_e3/common_e3.py) (Projection metrics), [`scripts/02b_search_alt_esgf_nodes.py`](https://github.com/japq91/smn_toe/blob/main/scripts/02b_search_alt_esgf_nodes.py) (Alternate ESGF nodes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_acquisition["Data acquisition"]
  node_scenario_config["Scenario configuration<br/>[pipeline_config.py]"]
  node_model_candidates["Candidate models"]
  node_esgf_catalog["ESGF catalog"]
  node_alternate_nodes["Alternate ESGF nodes"]
  node_cds_download["Copernicus download"]
end

subgraph group_prep["E1 preparation"]
  node_ersstv5_input["ERSSTv5 observations"]
  node_ocean_mask["Ocean masking"]
  node_quality_control["Quality control<br/>[06_qc_checks.py]"]
  node_inventory["Model inventory"]
  node_e1_plots["E1 diagnostic plots<br/>[plot_common.py]"]
end

subgraph group_historical["E2 historical evaluation"]
  node_e2_metrics["Historical metrics<br/>[common_e2.py]"]
  node_e2_events["Historical warm events<br/>[p07_eventos.py]"]
  node_e2_composites["Event composites"]
  node_e2_tests["Event tests"]
  node_e2_skill["Skill and selection<br/>[p09_skill_score.py]"]
  node_e2_plots["E2 figures<br/>[plot00_comun.py]"]
end

subgraph group_projection["E3 projections"]
  node_e3_analysis["Projection metrics<br/>[common_e3.py]"]
  node_e3_events["Future warm events"]
  node_e3_change_tests["Event change tests"]
  node_e3_plots["E3 figures<br/>[plot00_comun_e3.py]"]
end

node_researcher(("Research user"))
node_esgf_service{{"ESGF service"}}
node_cds_service{{"Copernicus CDS"}}

node_researcher -->|"starts workflow"| node_model_candidates
node_scenario_config -->|"configures experiments"| node_model_candidates
node_model_candidates -->|"queries availability"| node_esgf_service
node_model_candidates -.->|"provides candidates"| node_esgf_catalog
node_esgf_catalog -->|"searches files"| node_esgf_service
node_esgf_catalog -->|"unresolved files"| node_alternate_nodes
node_alternate_nodes -->|"searches mirrors"| node_esgf_service
node_alternate_nodes -->|"updates catalog"| node_esgf_catalog
node_alternate_nodes -->|"unresolved data"| node_cds_download
node_cds_download -.->|"downloads data"| node_cds_service
node_cds_download -->|"updates catalog"| node_esgf_catalog
node_esgf_catalog -.->|"prepared via E1"| node_ocean_mask
node_ersstv5_input -->|"joins preparation"| node_ocean_mask
node_ocean_mask -->|"masked data"| node_quality_control
node_quality_control -->|"QC output"| node_inventory
node_inventory -->|"selects plotted models"| node_e1_plots
node_ocean_mask -->|"supplies prepared data"| node_e2_metrics
node_e2_metrics -->|"provides indices"| node_e2_events
node_e2_events -->|"provides event dates"| node_e2_composites
node_e2_metrics -->|"provides indices"| node_e2_tests
node_e2_events -->|"provides events"| node_e2_tests
node_e2_composites -->|"provides Taylor scores"| node_e2_skill
node_e2_tests -.->|"filters models"| node_e2_skill
node_e2_metrics -->|"loads metrics"| node_e2_plots
node_e2_composites -->|"loads composites"| node_e2_plots
node_e2_skill -->|"selection informs projections"| node_e3_analysis
node_ocean_mask -->|"supplies prepared data"| node_e3_analysis
node_e3_analysis -->|"provides indices"| node_e3_events
node_e3_events -->|"provides events"| node_e3_change_tests
node_e3_analysis -->|"provides indices"| node_e3_change_tests
node_e3_analysis -->|"supplies projection data"| node_e3_plots
node_e3_events -.->|"supplies event results"| node_e3_plots

click node_scenario_config "https://github.com/japq91/smn_toe/blob/main/scripts/pipeline_config.py"
click node_model_candidates "https://github.com/japq91/smn_toe/blob/main/scripts/00b_build_model_list.py"
click node_esgf_catalog "https://github.com/japq91/smn_toe/blob/main/scripts/01_query_esgf_catalog.py"
click node_alternate_nodes "https://github.com/japq91/smn_toe/blob/main/scripts/02b_search_alt_esgf_nodes.py"
click node_cds_download "https://github.com/japq91/smn_toe/blob/main/scripts/02c_download_copernicus_cds.py"
click node_ocean_mask "https://github.com/japq91/smn_toe/blob/main/scripts/05_apply_ocean_mask.py"
click node_quality_control "https://github.com/japq91/smn_toe/blob/main/scripts/06_qc_checks.py"
click node_inventory "https://github.com/japq91/smn_toe/blob/main/scripts/07_build_inventory_report.py"
click node_e1_plots "https://github.com/japq91/smn_toe/blob/main/scripts/plot_common.py"
click node_e2_metrics "https://github.com/japq91/smn_toe/blob/main/scripts_e2/common_e2.py"
click node_e2_events "https://github.com/japq91/smn_toe/blob/main/scripts_e2/p07_eventos.py"
click node_e2_composites "https://github.com/japq91/smn_toe/blob/main/scripts_e2/p08_taylor_compuesto.py"
click node_e2_tests "https://github.com/japq91/smn_toe/blob/main/scripts_e2/p10_pruebas_eventos.py"
click node_e2_skill "https://github.com/japq91/smn_toe/blob/main/scripts_e2/p09_skill_score.py"
click node_e2_plots "https://github.com/japq91/smn_toe/blob/main/scripts_e2/plot00_comun.py"
click node_e3_analysis "https://github.com/japq91/smn_toe/blob/main/scripts_e3/common_e3.py"
click node_e3_events "https://github.com/japq91/smn_toe/blob/main/scripts_e3/q07_eventos_futuros.py"
click node_e3_change_tests "https://github.com/japq91/smn_toe/blob/main/scripts_e3/q08_cambio_eventos.py"
click node_e3_plots "https://github.com/japq91/smn_toe/blob/main/scripts_e3/plot00_comun_e3.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scenario_config,node_model_candidates,node_esgf_catalog,node_alternate_nodes,node_cds_download,node_researcher toneBlue
class node_ersstv5_input,node_ocean_mask,node_quality_control,node_inventory,node_e1_plots toneAmber
class node_e2_metrics,node_e2_events,node_e2_composites,node_e2_tests,node_e2_skill,node_e2_plots toneMint
class node_e3_analysis,node_e3_events,node_e3_change_tests,node_e3_plots toneRose
class node_esgf_service,node_cds_service toneIndigo
```

## Components

### Data acquisition

- [Scenario configuration](https://github.com/japq91/smn_toe/blob/main/scripts/pipeline_config.py): `scripts/pipeline_config.py`
- [Candidate models](https://github.com/japq91/smn_toe/blob/main/scripts/00b_build_model_list.py): `scripts/00b_build_model_list.py`
- [ESGF catalog](https://github.com/japq91/smn_toe/blob/main/scripts/01_query_esgf_catalog.py): `scripts/01_query_esgf_catalog.py`
- [Alternate ESGF nodes](https://github.com/japq91/smn_toe/blob/main/scripts/02b_search_alt_esgf_nodes.py): `scripts/02b_search_alt_esgf_nodes.py`
- [Copernicus download](https://github.com/japq91/smn_toe/blob/main/scripts/02c_download_copernicus_cds.py): `scripts/02c_download_copernicus_cds.py`

### E1 preparation

- **ERSSTv5 observations**
- [Ocean masking](https://github.com/japq91/smn_toe/blob/main/scripts/05_apply_ocean_mask.py): `scripts/05_apply_ocean_mask.py`
- [Quality control](https://github.com/japq91/smn_toe/blob/main/scripts/06_qc_checks.py): `scripts/06_qc_checks.py`
- [Model inventory](https://github.com/japq91/smn_toe/blob/main/scripts/07_build_inventory_report.py): `scripts/07_build_inventory_report.py`
- [E1 diagnostic plots](https://github.com/japq91/smn_toe/blob/main/scripts/plot_common.py): `scripts/plot_common.py`

### E2 historical evaluation

- [Historical metrics](https://github.com/japq91/smn_toe/blob/main/scripts_e2/common_e2.py): `scripts_e2/common_e2.py`
- [Historical warm events](https://github.com/japq91/smn_toe/blob/main/scripts_e2/p07_eventos.py): `scripts_e2/p07_eventos.py`
- [Event composites](https://github.com/japq91/smn_toe/blob/main/scripts_e2/p08_taylor_compuesto.py): `scripts_e2/p08_taylor_compuesto.py`
- [Event tests](https://github.com/japq91/smn_toe/blob/main/scripts_e2/p10_pruebas_eventos.py): `scripts_e2/p10_pruebas_eventos.py`
- [Skill and selection](https://github.com/japq91/smn_toe/blob/main/scripts_e2/p09_skill_score.py): `scripts_e2/p09_skill_score.py`
- [E2 figures](https://github.com/japq91/smn_toe/blob/main/scripts_e2/plot00_comun.py): `scripts_e2/plot00_comun.py`

### E3 projections

- [Projection metrics](https://github.com/japq91/smn_toe/blob/main/scripts_e3/common_e3.py): `scripts_e3/common_e3.py`
- [Future warm events](https://github.com/japq91/smn_toe/blob/main/scripts_e3/q07_eventos_futuros.py): `scripts_e3/q07_eventos_futuros.py`
- [Event change tests](https://github.com/japq91/smn_toe/blob/main/scripts_e3/q08_cambio_eventos.py): `scripts_e3/q08_cambio_eventos.py`
- [E3 figures](https://github.com/japq91/smn_toe/blob/main/scripts_e3/plot00_comun_e3.py): `scripts_e3/plot00_comun_e3.py`

### Other components

- **Research user**
- **ESGF service**
- **Copernicus CDS**

## Connections

- Research user → Candidate models: starts workflow
- Scenario configuration → Candidate models: configures experiments (evidence: `scripts/00b_build_model_list.py`)
- Candidate models → ESGF service: queries availability (evidence: `scripts/00b_build_model_list.py`)
- Candidate models → ESGF catalog: provides candidates
- ESGF catalog → ESGF service: searches files (evidence: `scripts/01_query_esgf_catalog.py`)
- ESGF catalog → Alternate ESGF nodes: unresolved files
- Alternate ESGF nodes → ESGF service: searches mirrors (evidence: `scripts/02b_search_alt_esgf_nodes.py`)
- Alternate ESGF nodes → ESGF catalog: updates catalog
- Alternate ESGF nodes → Copernicus download: unresolved data
- Copernicus download → Copernicus CDS: downloads data
- Copernicus download → ESGF catalog: updates catalog
- ESGF catalog → Ocean masking: prepared via E1
- ERSSTv5 observations → Ocean masking: joins preparation
- Ocean masking → Quality control: masked data
- Quality control → Model inventory: QC output
- Model inventory → E1 diagnostic plots: selects plotted models
- Ocean masking → Historical metrics: supplies prepared data
- Historical metrics → Historical warm events: provides indices (evidence: `scripts_e2/p07_eventos.py`)
- Historical warm events → Event composites: provides event dates (evidence: `scripts_e2/p08_taylor_compuesto.py`)
- Historical metrics → Event tests: provides indices (evidence: `scripts_e2/p10_pruebas_eventos.py`)
- Historical warm events → Event tests: provides events (evidence: `scripts_e2/p10_pruebas_eventos.py`)
- Event composites → Skill and selection: provides Taylor scores (evidence: `scripts_e2/p09_skill_score.py`)
- Event tests → Skill and selection: filters models
- Historical metrics → E2 figures: loads metrics (evidence: `scripts_e2/plot00_comun.py`)
- Event composites → E2 figures: loads composites (evidence: `scripts_e2/plot00_comun.py`)
- Skill and selection → Projection metrics: selection informs projections (evidence: `scripts_e3/common_e3.py`)
- Ocean masking → Projection metrics: supplies prepared data
- Projection metrics → Future warm events: provides indices (evidence: `scripts_e3/q07_eventos_futuros.py`)
- Future warm events → Event change tests: provides events (evidence: `scripts_e3/q08_cambio_eventos.py`)
- Projection metrics → Event change tests: provides indices (evidence: `scripts_e3/q08_cambio_eventos.py`)
- Projection metrics → E3 figures: supplies projection data (evidence: `scripts_e3/plot00_comun_e3.py`)
- Future warm events → E3 figures: supplies event results
