How davidebason/grid-stress works
Gridstress turns Dutch ENTSO-E price, load and generation responses into validated DuckDB tables, then analyzes when flexible electricity consumption can save money. The main workflow is fetch → parse/load → derive and validate → query analyses → report recommendations, with separate evidence and figure-generation branches. README and sampled loader/client code establish the principal handoffs; analysis and reporting details are documented at a higher level, so their internal query-to-query relationships are not inferred.
Main entry points: load.py Transactional loader · fetch.py Dataset fetcher · 04_shifting_gain.sql Shifting gains · client.py API client
Components 24
Data ingestion
- Dataset fetcher
src/gridstress/fetch.py - API client
src/gridstress/client.py - Raw XML
data/raw - XML parser
src/gridstress/my_parse.py
Data modeling
- Transactional loader
src/gridstress/load.py - Fact schema
sql/02_facts.sql - Dimensions
sql/01_dim_production_type.sql - Time dimension
sql/03_dim_time.sql - Derived tables
sql/04_hourly_tables.sql - Post-crisis tables
sql/05_post_crisis.sql - Price-unit table
sql/06_price_units.sql - DuckDB warehouse
- Data validation
src/gridstress/validate.py
Analysis
- Price extremes
analysis/01_negative_and_extreme.sql - Generation mix
analysis/02_generation_mix.sql - Generation swing
analysis/03_swing.sql - Shifting gains
analysis/04_shifting_gain.sql
Evidence and outputs
- Evidence checks
evidence - Findings and advice
REPORT.md - Data provenance
DATA.md - Figure generation
notebooks/image_creation.ipynb - Savings figure
figures
Other components
- Analyst
- ENTSO-E API
Connections 27
- Analyst → Dataset fetcher: starts fetch
README.md - Dataset fetcher → API client: requests windows
src/gridstress/fetch.py - API client → ENTSO-E API: requests data
README.md - Dataset fetcher → Raw XML: saves responses
src/gridstress/fetch.py - Raw XML → Transactional loader: supplies files
src/gridstress/load.py - Transactional loader → XML parser: parses responses
src/gridstress/load.py - Transactional loader → Dimensions: runs schema
src/gridstress/load.py - Transactional loader → Fact schema: runs schema
src/gridstress/load.py - Transactional loader → Time dimension: runs schema
src/gridstress/load.py - Transactional loader → DuckDB warehouse: inserts facts
src/gridstress/load.py - Transactional loader → Derived tables: executes SQL
src/gridstress/load.py - Transactional loader → Post-crisis tables: executes SQL
src/gridstress/load.py - Transactional loader → Price-unit table: executes SQL
src/gridstress/load.py - Transactional loader → Data validation: validates transaction
src/gridstress/load.py - Data validation → DuckDB warehouse: checks tables
src/gridstress/validate.py - Analyst → Price extremes: runs query
README.md - Analyst → Generation mix: runs query
README.md - Analyst → Generation swing: runs query
README.md - Analyst → Shifting gains: runs query
README.md - Price extremes → DuckDB warehouse: queries tables
README.md - Generation mix → DuckDB warehouse: queries tables
README.md - Generation swing → DuckDB warehouse: queries tables
README.md - Shifting gains → DuckDB warehouse: queries tables
README.md - Evidence checks → Data provenance: regenerates figures
README.md - Analyst → Findings and advice: reads findings
README.md - Figure generation → Savings figure: draws figure
README.md - Shifting gains → Findings and advice: informs recommendation