# fzj-iek3-vsa/fine architecture

> Architecture diagram of the GitHub repository fzj-iek3-vsa/fine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fzj-iek3-vsa/fine
- Repository: https://github.com/fzj-iek3-vsa/fine

## Overview

#### Purpose and entry points
ETHOS.FINE is a Python library for cost-optimal, multi-region and multi-commodity energy-system design and operation. Users construct an energy-system model, add technology components, optimize through Pyomo, then inspect/export results. The public package root is `fine/__init__.py`; the orchestration entry point is `fine/energySystemModel.py`. Executable usage is primarily notebook/script based, e.g. `examples/00_Tutorial/00_Tutorial.ipynb` and `examples/01_1node_Energy_System_Workflow/getData.py`.

#### Model orchestration and optimization boundary
`EnergySystemModel` owns global model dimensions—locations, commodities, time steps, and investment periods—and coordinates component registration, constraint assembly, optimization, and result handling. It is the boundary between FINE’s domain model and the Pyomo optimization model. Optimization minimizes NPV (or TAC for one investment period), subject to technical/environmental constraints.
Paths: `fine/energySystemModel.py`, `fine/component.py`, `fine/enums.py`.

#### Component model hierarchy
The component framework provides shared parameter validation, sets, variables, constraints, and result interfaces; concrete component families formulate their own Pyomo contributions. Sources/sinks represent supply and demand, conversions transform commodities, storage couples charge/discharge over time, and transmission links locations. Specialized subclasses add dynamic conversion, part-load behavior, and linear optimal power flow.
Paths: `fine/component.py`, `fine/sourceSink.py`, `fine/conversion.py`, `fine/storage.py`, `fine/transmission.py`, `fine/subclasses/lopf.py`.

#### Primary execution flow
A user supplies scalar, tabular, temporal, and spatial inputs; instantiates `EnergySystemModel`; creates and adds components; optionally reduces/aggregates inputs; then invokes model optimization against a selected solver. Each component injects its indexed decision variables and constraints into the central Pyomo problem. Solved capacities, operations, costs, and shadow-price-related outputs are collected back into component/model result structures.
Paths: `fine/energySystemModel.py`, `fine/component.py`, `fine/utils.py`.

#### Temporal, pathway, and uncertainty extensions
The core supports full-resolution time series and temporal aggregation via typical periods. Extension modules implement multi-stage temporal aggregation, transformation-path/perfect-foresight optimization across investment periods, and piecewise-linear cost formulations. Stochastic and pathway-oriented workflows are exposed through the same central model/component abstraction rather than a separate service.
Paths: `fine/expansionModules/optimizeTSAmultiStage.py`, `fine/expansionModules/transformationPath.py`, `fine/expansionModules/piecewiseLinearCostFunction.py`.

#### Input/output and reproducibility
I/O adapters bridge the in-memory model to dictionary, conventional tabular/Excel-oriented, and xarray/netCDF representations. This is a material boundary: external datasets are normalized into FINE inputs, while full model inputs/results can be persisted reproducibly as netCDF. Example data includes Excel workbooks, netCDF, and GIS shapefiles.
Paths: `fine/IOManagement/dictIO.py`, `fine/IOManagement/standardIO.py`, `fine/IOManagement/xarrayIO.py`.

#### Model-size reduction
Spatial aggregation groups regions and adapts associated model inputs, including network/spatial data; technology aggregation consolidates similar technologies. These preprocessing capabilities reduce optimization dimensionality before the central model is formulated.
Paths: `fine/aggregations/spatialAggregation/manager.py`, `fine/aggregations/spatialAggregation/aggregation.py`, `fine/aggregations/technologyAggregation/techAggregation.py`.

#### Runtime and infrastructure
Runtime is Python, packaged via `pyproject.toml`, with Pyomo as the optimization interface. A MILP solver is required; supported/tested solver integrations include Gurobi, HiGHS, and GLPK. The project is distributed through PyPI and conda-forge, containerized by `Dockerfile`, and uses GitHub Actions/GitLab CI for validation. Documentation is MkDocs/Read the Docs based.
Paths: `pyproject.toml`, `Dockerfile`, `.github/workflows/test_push.yml`, `mkdocs.yml`.

Main entry points: [`fine/energySystemModel.py`](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py) (EnergySystemModel), [`fine/component.py`](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/component.py) (Component framework), [`examples/01_1node_Energy_System_Workflow/getData.py`](https://github.com/fzj-iek3-vsa/fine/blob/develop/examples/01_1node_Energy_System_Workflow/getData.py) (Energy-system inputs), [`fine/IOManagement/xarrayIO.py`](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/IOManagement/xarrayIO.py) (I/O adapters)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inputs["Data & preprocessing"]
  node_input_data["Energy-system inputs<br/>[getData.py]"]
  node_spatial_aggregation["Spatial aggregation<br/>preprocessing<br/>[manager.py]"]
  node_technology_aggregation["Technology aggregation<br/>preprocessing<br/>[techAggregation.py]"]
end

subgraph group_core["Modeling core"]
  node_energy_system_model["EnergySystemModel<br/>model orchestrator"]
  node_component_framework["Component framework<br/>shared formulation base<br/>[component.py]"]
  node_enums["Model enums<br/>domain definitions<br/>[enums.py]"]
  node_pyomo_problem{{"Pyomo optimization problem<br/>MILP formulation"}}
end

subgraph group_components["Component formulations"]
  node_source_sink["Source / sink<br/>component family<br/>[sourceSink.py]"]
  node_conversion["Conversion<br/>component family<br/>[conversion.py]"]
  node_storage["Storage<br/>component family<br/>[storage.py]"]
  node_transmission["Transmission &amp; LOPF<br/>network component<br/>[transmission.py]"]
end

subgraph group_extensions["Extensions"]
  node_extensions["Temporal &amp; pathway extensions"]
end

subgraph group_io["Persistence &amp; exchange"]
  node_io_adapters["I/O adapters<br/>[xarrayIO.py]"]
  node_results["Model results<br/>capacities, operations &amp; costs"]
end

node_user["Notebook / script user<br/>usage<br/>[00_Tutorial.ipynb]"]
node_solver["MILP solver<br/>[pyproject.toml]"]

node_user -->|"supplies"| node_input_data
node_user -->|"constructs"| node_energy_system_model
node_input_data -.->|"optionally reduced"| node_spatial_aggregation
node_input_data -.->|"optionally consolidated"| node_technology_aggregation
node_spatial_aggregation -->|"reduced inputs"| node_energy_system_model
node_technology_aggregation -->|"reduced inputs"| node_energy_system_model
node_input_data -->|"import / export"| node_io_adapters
node_io_adapters -->|"normalized inputs"| node_energy_system_model
node_energy_system_model -->|"registers components"| node_component_framework
node_energy_system_model -->|"uses dimensions"| node_enums
node_component_framework -->|"base for"| node_source_sink
node_component_framework -->|"base for"| node_conversion
node_component_framework -->|"base for"| node_storage
node_component_framework -->|"base for"| node_transmission
node_source_sink -->|"adds variables &amp; constraints"| node_pyomo_problem
node_conversion -->|"adds variables &amp; constraints"| node_pyomo_problem
node_storage -->|"adds variables &amp; constraints"| node_pyomo_problem
node_transmission -->|"adds variables &amp; constraints"| node_pyomo_problem
node_extensions -.->|"extends workflows"| node_energy_system_model
node_energy_system_model -->|"assembles"| node_pyomo_problem
node_pyomo_problem -->|"solves with"| node_solver
node_solver -->|"solution"| node_results
node_energy_system_model -->|"collects"| node_results
node_results -->|"persist / export"| node_io_adapters

click node_user "https://github.com/fzj-iek3-vsa/fine/blob/develop/examples/00_Tutorial/00_Tutorial.ipynb"
click node_input_data "https://github.com/fzj-iek3-vsa/fine/blob/develop/examples/01_1node_Energy_System_Workflow/getData.py"
click node_spatial_aggregation "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/aggregations/spatialAggregation/manager.py"
click node_technology_aggregation "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/aggregations/technologyAggregation/techAggregation.py"
click node_energy_system_model "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py"
click node_component_framework "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/component.py"
click node_enums "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/enums.py"
click node_source_sink "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/sourceSink.py"
click node_conversion "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/conversion.py"
click node_storage "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/storage.py"
click node_transmission "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/transmission.py"
click node_pyomo_problem "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py"
click node_solver "https://github.com/fzj-iek3-vsa/fine/blob/develop/pyproject.toml"
click node_extensions "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/expansionModules/optimizeTSAmultiStage.py"
click node_io_adapters "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/IOManagement/xarrayIO.py"
click node_results "https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_input_data,node_spatial_aggregation,node_technology_aggregation toneBlue
class node_energy_system_model,node_component_framework,node_enums,node_pyomo_problem toneAmber
class node_source_sink,node_conversion,node_storage,node_transmission toneMint
class node_extensions toneRose
class node_io_adapters,node_results toneIndigo
class node_user,node_solver toneNeutral
```

## Components

### Data & preprocessing

- [Energy-system inputs](https://github.com/fzj-iek3-vsa/fine/blob/develop/examples/01_1node_Energy_System_Workflow/getData.py): tabular, temporal & spatial data · `examples/01_1node_Energy_System_Workflow/getData.py`
- [Spatial aggregation](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/aggregations/spatialAggregation/manager.py): preprocessing · Groups regions and adapts spatial and network inputs. · `fine/aggregations/spatialAggregation/manager.py`
- [Technology aggregation](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/aggregations/technologyAggregation/techAggregation.py): preprocessing · `fine/aggregations/technologyAggregation/techAggregation.py`

### Modeling core

- [EnergySystemModel](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py): model orchestrator · Owns dimensions, registration, formulation, optimization, and results. · `fine/energySystemModel.py`
- [Component framework](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/component.py): shared formulation base · Provides shared validation, sets, variables, constraints, and result interfaces. · `fine/component.py`
- [Model enums](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/enums.py): domain definitions · `fine/enums.py`
- [Pyomo optimization problem](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py): MILP formulation · Minimizes NPV or TAC under technical and environmental constraints. · `fine/energySystemModel.py`

### Component formulations

- [Source / sink](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/sourceSink.py): component family · Models commodity supply and demand. · `fine/sourceSink.py`
- [Conversion](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/conversion.py): component family · Transforms commodities, including specialized conversion behavior. · `fine/conversion.py`
- [Storage](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/storage.py): component family · Couples charging and discharging across time. · `fine/storage.py`
- [Transmission & LOPF](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/transmission.py): network component · Links locations and supports linear optimal power flow. · `fine/transmission.py`

### Extensions

- [Temporal & pathway extensions](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/expansionModules/optimizeTSAmultiStage.py): aggregation, foresight & PWL costs · Adds multi-stage TSA, transformation paths, and piecewise-linear costs. · `fine/expansionModules/optimizeTSAmultiStage.py`

### Persistence & exchange

- [I/O adapters](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/IOManagement/xarrayIO.py): dict, tabular & xarray I/O · Imports normalized inputs and persists models and results as netCDF. · `fine/IOManagement/xarrayIO.py`
- [Model results](https://github.com/fzj-iek3-vsa/fine/blob/develop/fine/energySystemModel.py): capacities, operations & costs · Includes shadow-price-related outputs. · `fine/energySystemModel.py`

### Other components

- [Notebook / script user](https://github.com/fzj-iek3-vsa/fine/blob/develop/examples/00_Tutorial/00_Tutorial.ipynb): usage · `examples/00_Tutorial/00_Tutorial.ipynb`
- [MILP solver](https://github.com/fzj-iek3-vsa/fine/blob/develop/pyproject.toml): Gurobi / HiGHS / GLPK · `pyproject.toml`

## Connections

- Notebook / script user → Energy-system inputs: supplies
- Notebook / script user → EnergySystemModel: constructs
- Energy-system inputs → Spatial aggregation: optionally reduced
- Energy-system inputs → Technology aggregation: optionally consolidated
- Spatial aggregation → EnergySystemModel: reduced inputs
- Technology aggregation → EnergySystemModel: reduced inputs
- Energy-system inputs → I/O adapters: import / export
- I/O adapters → EnergySystemModel: normalized inputs
- EnergySystemModel → Component framework: registers components
- EnergySystemModel → Model enums: uses dimensions
- Component framework → Source / sink: base for
- Component framework → Conversion: base for
- Component framework → Storage: base for
- Component framework → Transmission & LOPF: base for
- Source / sink → Pyomo optimization problem: adds variables & constraints
- Conversion → Pyomo optimization problem: adds variables & constraints
- Storage → Pyomo optimization problem: adds variables & constraints
- Transmission & LOPF → Pyomo optimization problem: adds variables & constraints
- Temporal & pathway extensions → EnergySystemModel: extends workflows
- EnergySystemModel → Pyomo optimization problem: assembles
- Pyomo optimization problem → MILP solver: solves with
- MILP solver → Model results: solution
- EnergySystemModel → Model results: collects
- Model results → I/O adapters: persist / export