How baptistefeldmann/observatoire-secheresse works
This repository implements a configurable, weekly departmental drought observatory. Its main workflow builds geographic and station reference data, ingests hydrological, groundwater, ONDE, reservoir, and weather observations, derives reference normals and weekly indices, then persists results for downstream use, including PostGIS loading. The graph reflects documented and indexed module relationships; API and dashboard are explicitly future work, and Sentinel-2 is described as a later capability, without a sampled implementation to connect.
Main entry points: referentiels.py Reference builder · __main__.py Pipeline CLI · ingestion.py Observation ingestion · __init__.py Weekly indices
Components 27
Configuration
- Pipeline CLI
pipeline/__main__.py - Config validation
pipeline/config.py
Data acquisition
- HTTP client
pipeline/http.py - Reference builder
pipeline/referentiels.py - Zones and station links
pipeline/zonage.py - SANDRE data
pipeline/sources/sandre.py - Commune data
pipeline/sources/communes.py - Weather and SIM
pipeline/sources/meteo.py - Hydro and ONDE stations
pipeline/sources/hydro.py - Groundwater stations
pipeline/sources/piezo.py - Reservoir sources
pipeline/sources/retenues.py - Observation ingestion
pipeline/ingestion.py - Hub’Eau helpers
pipeline/sources/hubeau.py
Drought analytics
- Reference normals
pipeline/reference - Normal helpers
pipeline/reference/commun.py - Flow normals
pipeline/reference/debit.py - SPI normals
pipeline/reference/spi.py - Weekly indices
pipeline/indices/__init__.py - Index utilities
pipeline/indices/commun.py - Index components
pipeline/indices
Storage and access
- Parquet storage
pipeline/stockage.py - Data schemas
pipeline/schema.py - Raw observations
data - Index outputs
data - PostGIS loader
pipeline/db/chargement.py - PostGIS database
Other components
- Operator
Connections 42
- Operator → Pipeline CLI: runs
README.md - Pipeline CLI → Config validation: loads
pipeline/__main__.py - Pipeline CLI → Reference builder: builds references
pipeline/__main__.py - Pipeline CLI → Observation ingestion: runs ingestion
pipeline/__main__.py - Pipeline CLI → Reference normals: builds normals
pipeline/__main__.py - Pipeline CLI → Weekly indices: calculates indices
pipeline/__main__.py - Pipeline CLI → PostGIS loader: loads database
pipeline/__main__.py - Reference builder → Commune data: fetches
pipeline/referentiels.py - Reference builder → SANDRE data: fetches
pipeline/referentiels.py - Reference builder → Weather and SIM: fetches
pipeline/referentiels.py - Reference builder → Hydro and ONDE stations: fetches
pipeline/referentiels.py - Reference builder → Groundwater stations: fetches
pipeline/referentiels.py - Reference builder → Reservoir sources: fetches
pipeline/referentiels.py - Reference builder → Zones and station links: constructs zones
pipeline/referentiels.py - Reference builder → Parquet storage: writes references
pipeline/referentiels.py - Observation ingestion → Reference builder: reads references
pipeline/ingestion.py - Observation ingestion → Groundwater stations: ingests observations
pipeline/ingestion.py - Observation ingestion → Hydro and ONDE stations: ingests observations
pipeline/ingestion.py - Observation ingestion → Weather and SIM: ingests observations
pipeline/ingestion.py - Observation ingestion → Reservoir sources: ingests observations
pipeline/ingestion.py - Observation ingestion → Parquet storage: stores observations
pipeline/ingestion.py - Groundwater stations → Hub’Eau helpers: uses helpers
pipeline/sources/piezo.py - Hydro and ONDE stations → Hub’Eau helpers: uses helpers
pipeline/sources/hydro.py - Groundwater stations → HTTP client: requests data
pipeline/sources/piezo.py - Hydro and ONDE stations → HTTP client: requests data
pipeline/sources/hydro.py - Weather and SIM → HTTP client: requests data
pipeline/sources/meteo.py - Reservoir sources → HTTP client: requests data
pipeline/sources/retenues.py - Commune data → HTTP client: requests data
pipeline/sources/communes.py - SANDRE data → HTTP client: requests data
pipeline/sources/sandre.py - Reference normals → Normal helpers: uses helpers
pipeline/reference/__init__.py - Flow normals → Normal helpers: uses helpers
pipeline/reference/debit.py - SPI normals → Normal helpers: uses helpers
pipeline/reference/spi.py - Reference normals → Parquet storage: reads raw data
pipeline/reference/commun.py - Weekly indices → Index utilities: uses utilities
pipeline/indices/__init__.py - Weekly indices → Index components: dispatches calculations
pipeline/indices/__init__.py - Weekly indices → Reference normals: reads normals
pipeline/indices/__init__.py - Weekly indices → Parquet storage: reads observations
pipeline/indices/__init__.py - Weekly indices → Data schemas: normalizes outputs
pipeline/indices/__init__.py - Weekly indices → Parquet storage: writes indices
pipeline/indices/__init__.py - Parquet storage → Raw observations: persists observations
- Parquet storage → Index outputs: persists indices
- PostGIS loader → PostGIS database: loads tables
README.md