How pypsa/pypsa-eur works
Project Type and Purpose
• The project is a large-scale energy system modelling tool designed to simulate and analyse the European transmission network. It supports both operational studies and long-term generation and transmission expansion planning.
• The model ingests real-world grid data (from ENTSO-E and GridKit), renewable and demand time series (from OPSD, ERA5, atlite), and also includes an extensive sector-coupled extension (covering transport, industry, heating, etc.).
Main Components
a. Data and Configuration:
– Data Directory: Represents various CSV, JSON, and YAML datasets (e.g., grid data, demand, potentials).
– Configuration Files: The “config” folder (with default YAML settings, scenario templates, and plotting configurations) that steer model parameters and scenario setups.
b. Build and Preprocessing Modules:
– Scripts in the “scripts” directory are responsible for multiple tasks such as building power plants, processing power and heat demand profiles, clustering network nodes, and preparing geographic as well as temporal data.
– The “rules” directory holds the Snakemake workflow scripts that orchestrate these tasks (like data retrieval, network building, and post-processing).
c. Solve (Optimisation) Modules:
– Although a custom note mentioned “i miss the solve scripts”, the project actually contains multiple solving routines:
• In the “scripts” folder: files such as “solve_network.py” and “solve_operations_network.py” perform the optimisation routines.
• In the “rules” folder: several Snakemake rules (e.g., “solve_electricity.smk”, “solve_myopic.smk”, “solve_overnight.smk”, “solve_perfect.smk”) coordinate the execution of different optimisation methods.
– These solving components are vital as they execute the network optimisation using the PyPSA framework.
d. Documentation, Testing, and Deployment:
– GitHub workflows, Docker files, and pre-commit configurations support CI/CD and development best practices.
– “test” directory contains various tests to validate components like the base network and building routines.
Relationships and Interactions
• Data flows start from raw data (in “data”) and configuration files (in “config”) into data processing scripts in “scripts” and Snakemake rules. These modules preprocess and cluster data for the model.
• Preprocessed data is then passed to the core network building components (also part of “scripts” and “rules”) which integrate with PyPSA as the central energy system model.
• The solve scripts (both stand-alone and via Snakemake rules) take the fully built network model and perform optimisation, determining optimal power flows and generation capacities.
• Postprocessing scripts generate plots, summaries, and tables that are eventually documented (in “doc”) and tested (in “test”).
Key Architectural Patterns and Design Principles
• Modular Design: Clear separation between data ingestion, model building, solution (optimisation) routines, and postprocessing/visualization.
• Workflow Orchestration: Use of Snakemake (found in the “rules” folder) to manage dependencies and steps, supporting reproducibility.
• Extensibility: Structured directory layout (e.g., separate “scripts” and “config” folders) that allows extending or modifying certain aspects (like adding new solving methods or data sources).
• Maintainability: The use of tests, CI workflows, and detailed documentation emphasize sustainable development practices.