How umr-cnrm/vortex works
Vortex is a Python library for defining individual tasks in numerical-weather-prediction pipelines. Its principal workflow is to declare input and output resources, retrieve inputs into task containers, execute a program (potentially across workers), and publish outputs. The graph includes the core resource, context, execution, and system layers plus documented NWP-specific algorithms and data. Arrows are limited to relationships shown in excerpts, the source index, or the README; NWP capability branches are included without speculative implementation links.
Main entry points: components.py Algorithm components · handlers.py Resource handlers · toolbox.py Task API · dataflow.py Dataflow tracking
Components 21
Task execution
- Task API
src/vortex/toolbox.py - Algorithm components
src/vortex/algo/components.py - Parallel execution
src/vortex/tools/parallelism.py
Resource handling
- Resource handlers
src/vortex/data/handlers.py - Data containers
src/vortex/data/containers.py - Resource contents
src/vortex/data/contents.py - Resource providers
src/vortex/data/providers.py - Storage backends
src/vortex/data/stores.py - Storage operations
src/vortex/tools/storage.py
Runtime context
- Dataflow tracking
src/vortex/layout/dataflow.py - Execution contexts
src/vortex/layout/contexts.py - Session tickets
src/vortex/sessions.py - Environment
src/vortex/tools/env.py - System interface
src/vortex/tools/systems.py - Execution monitoring
src/vortex/layout/monitor.py
NWP capabilities
- NWP algorithms
src/vortex/nwp/algo - NWP data types
src/vortex/nwp/data - NWP attributes
src/vortex/nwp/syntax/stdattrs.py - NWP tools
src/vortex/nwp/tools
Other components
- Task author
- Program / pipeline
Connections 23
- Task author → Task API: defines tasks
- Task API → Resource handlers: declares resources
src/vortex/toolbox.py - Task API → Dataflow tracking: uses dataflow
src/vortex/toolbox.py - Resource handlers → Data containers: feeds containers
src/vortex/data/handlers.py - Resource handlers → Resource providers: selects provider
src/vortex/data/handlers.py - Resource handlers → Storage backends: retrieves / publishes
src/vortex/data/handlers.py - Resource handlers → Dataflow tracking: updates tracking
src/vortex/data/handlers.py - Data containers → Resource contents: holds contents
src/vortex/data/__init__.py - Storage backends → Storage operations: uses storage
- Task API → Algorithm components: builds components
- Algorithm components → Program / pipeline: executes program
- Algorithm components → Parallel execution: dispatches work
src/vortex/algo/components.py - Session tickets → Execution contexts: tracks contexts
src/vortex/sessions.py - Execution contexts → Dataflow tracking: creates tracker
src/vortex/layout/contexts.py - Session tickets → Environment: provides environment
src/vortex/sessions.py - Session tickets → System interface: provides system
src/vortex/sessions.py - Algorithm components → Execution monitoring: monitors execution
- NWP algorithms → Algorithm components: specializes algorithms
src/vortex/nwp/algo/fpserver.py - NWP data types → Resource contents: extends resource types
src/vortex/nwp/data/obs.py - NWP data types → NWP attributes: uses attributes
src/vortex/nwp/data/obs.py - NWP tools → Algorithm components: uses components
src/vortex/nwp/tools/grib.py - NWP tools → Dataflow tracking: uses dataflow
src/vortex/nwp/tools/odb.py - Program / pipeline → Task API: supplies task logic