# umr-cnrm/vortex architecture

> Architecture diagram of the GitHub repository umr-cnrm/vortex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/umr-cnrm/vortex
- Repository: https://github.com/umr-cnrm/vortex

## Overview

Vortex is a Python library for defining individual tasks in numerical-weather-prediction pipelines. Its principal workflow is to declare input and output resources, retrieve inputs into task containers, execute a program (potentially across workers), and publish outputs. The graph includes the core resource, context, execution, and system layers plus documented NWP-specific algorithms and data. Arrows are limited to relationships shown in excerpts, the source index, or the README; NWP capability branches are included without speculative implementation links.

Main entry points: [`src/vortex/algo/components.py`](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/algo/components.py) (Algorithm components), [`src/vortex/data/handlers.py`](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/handlers.py) (Resource handlers), [`src/vortex/toolbox.py`](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/toolbox.py) (Task API), [`src/vortex/layout/dataflow.py`](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/dataflow.py) (Dataflow tracking)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_task["Task execution"]
  node_api["Task API<br/>[toolbox.py]"]
  node_algo["Algorithm components<br/>[components.py]"]
  node_parallel["Parallel execution<br/>[parallelism.py]"]
end

subgraph group_data["Resource handling"]
  node_handlers["Resource handlers<br/>[handlers.py]"]
  node_containers["Data containers<br/>[containers.py]"]
  node_contents["Resource contents<br/>[contents.py]"]
  node_providers["Resource providers<br/>[providers.py]"]
  node_stores["Storage backends<br/>[stores.py]"]
  node_storage["Storage operations<br/>[storage.py]"]
end

subgraph group_runtime["Runtime context"]
  node_dataflow["Dataflow tracking<br/>[dataflow.py]"]
  node_contexts["Execution contexts<br/>[contexts.py]"]
  node_sessions["Session tickets<br/>[sessions.py]"]
  node_environment["Environment<br/>[env.py]"]
  node_systems["System interface<br/>[systems.py]"]
  node_monitor["Execution monitoring<br/>[monitor.py]"]
end

subgraph group_nwp["NWP capabilities"]
  node_nwp_algo["NWP algorithms"]
  node_nwp_data["NWP data types"]
  node_nwp_syntax["NWP attributes<br/>[stdattrs.py]"]
  node_nwp_tools["NWP tools"]
end

node_user(("Task author"))
node_user_program["Program / pipeline"]

node_user -->|"defines tasks"| node_api
node_api -->|"declares resources"| node_handlers
node_api -->|"uses dataflow"| node_dataflow
node_handlers -->|"feeds containers"| node_containers
node_handlers -->|"selects provider"| node_providers
node_handlers -->|"retrieves / publishes"| node_stores
node_handlers -->|"updates tracking"| node_dataflow
node_containers -->|"holds contents"| node_contents
node_stores -.->|"uses storage"| node_storage
node_api -.->|"builds components"| node_algo
node_algo -->|"executes program"| node_user_program
node_algo -->|"dispatches work"| node_parallel
node_sessions -->|"tracks contexts"| node_contexts
node_contexts -->|"creates tracker"| node_dataflow
node_sessions -->|"provides environment"| node_environment
node_sessions -->|"provides system"| node_systems
node_algo -.->|"monitors execution"| node_monitor
node_nwp_algo -->|"specializes algorithms"| node_algo
node_nwp_data -->|"extends resource types"| node_contents
node_nwp_data -->|"uses attributes"| node_nwp_syntax
node_nwp_tools -->|"uses components"| node_algo
node_nwp_tools -->|"uses dataflow"| node_dataflow
node_user_program -->|"supplies task logic"| node_api

click node_api "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/toolbox.py"
click node_algo "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/algo/components.py"
click node_parallel "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/parallelism.py"
click node_handlers "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/handlers.py"
click node_containers "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/containers.py"
click node_contents "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/contents.py"
click node_providers "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/providers.py"
click node_stores "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/stores.py"
click node_storage "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/storage.py"
click node_dataflow "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/dataflow.py"
click node_contexts "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/contexts.py"
click node_sessions "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/sessions.py"
click node_environment "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/env.py"
click node_systems "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/systems.py"
click node_monitor "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/monitor.py"
click node_nwp_algo "https://github.com/umr-cnrm/vortex/tree/main/src/vortex/nwp/algo"
click node_nwp_data "https://github.com/umr-cnrm/vortex/tree/main/src/vortex/nwp/data"
click node_nwp_syntax "https://github.com/umr-cnrm/vortex/blob/main/src/vortex/nwp/syntax/stdattrs.py"
click node_nwp_tools "https://github.com/umr-cnrm/vortex/tree/main/src/vortex/nwp/tools"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_algo,node_parallel toneBlue
class node_handlers,node_containers,node_contents,node_providers,node_stores,node_storage toneAmber
class node_dataflow,node_contexts,node_sessions,node_environment,node_systems,node_monitor toneMint
class node_nwp_algo,node_nwp_data,node_nwp_syntax,node_nwp_tools,node_user toneRose
class node_user_program toneIndigo
```

## Components

### Task execution

- [Task API](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/toolbox.py): `src/vortex/toolbox.py`
- [Algorithm components](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/algo/components.py): `src/vortex/algo/components.py`
- [Parallel execution](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/parallelism.py): `src/vortex/tools/parallelism.py`

### Resource handling

- [Resource handlers](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/handlers.py): `src/vortex/data/handlers.py`
- [Data containers](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/containers.py): `src/vortex/data/containers.py`
- [Resource contents](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/contents.py): `src/vortex/data/contents.py`
- [Resource providers](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/providers.py): `src/vortex/data/providers.py`
- [Storage backends](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/data/stores.py): `src/vortex/data/stores.py`
- [Storage operations](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/storage.py): `src/vortex/tools/storage.py`

### Runtime context

- [Dataflow tracking](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/dataflow.py): `src/vortex/layout/dataflow.py`
- [Execution contexts](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/contexts.py): `src/vortex/layout/contexts.py`
- [Session tickets](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/sessions.py): `src/vortex/sessions.py`
- [Environment](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/env.py): `src/vortex/tools/env.py`
- [System interface](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/tools/systems.py): `src/vortex/tools/systems.py`
- [Execution monitoring](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/layout/monitor.py): `src/vortex/layout/monitor.py`

### NWP capabilities

- [NWP algorithms](https://github.com/umr-cnrm/vortex/tree/main/src/vortex/nwp/algo): `src/vortex/nwp/algo`
- [NWP data types](https://github.com/umr-cnrm/vortex/tree/main/src/vortex/nwp/data): `src/vortex/nwp/data`
- [NWP attributes](https://github.com/umr-cnrm/vortex/blob/main/src/vortex/nwp/syntax/stdattrs.py): `src/vortex/nwp/syntax/stdattrs.py`
- [NWP tools](https://github.com/umr-cnrm/vortex/tree/main/src/vortex/nwp/tools): `src/vortex/nwp/tools`

### Other components

- **Task author**
- **Program / pipeline**

## Connections

- Task author → Task API: defines tasks
- Task API → Resource handlers: declares resources (evidence: `src/vortex/toolbox.py`)
- Task API → Dataflow tracking: uses dataflow (evidence: `src/vortex/toolbox.py`)
- Resource handlers → Data containers: feeds containers (evidence: `src/vortex/data/handlers.py`)
- Resource handlers → Resource providers: selects provider (evidence: `src/vortex/data/handlers.py`)
- Resource handlers → Storage backends: retrieves / publishes (evidence: `src/vortex/data/handlers.py`)
- Resource handlers → Dataflow tracking: updates tracking (evidence: `src/vortex/data/handlers.py`)
- Data containers → Resource contents: holds contents (evidence: `src/vortex/data/__init__.py`)
- Storage backends → Storage operations: uses storage
- Task API → Algorithm components: builds components
- Algorithm components → Program / pipeline: executes program
- Algorithm components → Parallel execution: dispatches work (evidence: `src/vortex/algo/components.py`)
- Session tickets → Execution contexts: tracks contexts (evidence: `src/vortex/sessions.py`)
- Execution contexts → Dataflow tracking: creates tracker (evidence: `src/vortex/layout/contexts.py`)
- Session tickets → Environment: provides environment (evidence: `src/vortex/sessions.py`)
- Session tickets → System interface: provides system (evidence: `src/vortex/sessions.py`)
- Algorithm components → Execution monitoring: monitors execution
- NWP algorithms → Algorithm components: specializes algorithms (evidence: `src/vortex/nwp/algo/fpserver.py`)
- NWP data types → Resource contents: extends resource types (evidence: `src/vortex/nwp/data/obs.py`)
- NWP data types → NWP attributes: uses attributes (evidence: `src/vortex/nwp/data/obs.py`)
- NWP tools → Algorithm components: uses components (evidence: `src/vortex/nwp/tools/grib.py`)
- NWP tools → Dataflow tracking: uses dataflow (evidence: `src/vortex/nwp/tools/odb.py`)
- Program / pipeline → Task API: supplies task logic
