# matrixeigs/energy_management_system architecture

> Architecture diagram of the GitHub repository matrixeigs/energy_management_system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matrixeigs/energy_management_system
- Repository: https://github.com/matrixeigs/energy_management_system

## Overview

##### Project Type and Purpose
- This is a specialized full-stack computational platform—“Universal Energy Management System (EMS)”—for coordinating multiple hybrid AC/DC microgrids (MGs) in a microgrid park.
- It ingests data (historical, web-scraped, weather), applies forecasting, then sequentially drives unit commitment, optimal power flow, and economic dispatch, using configurable models and solvers. A local EMS manages individual MGs, and a universal EMS orchestrates at park level.

##### High-Level Architecture Components
a. Entry & Orchestration
- universal_ems.py (top-level orchestration)
- start_up/start_up_uems.py and start_up/start_up_lems.py (bootstrapping universal & local EMS instances)
b. Configuration
- configuration/ (all configuration loaders, defaults, database connection, converters)
c. Data Ingestion & Management
- data_management/ (SQL scripts, database management, information collection and web_scraping)
- External data sources (weather station, historical data files)
- Central time-series database
d. Modelling Library
- modelling/ (Protobuf schemas, generators, loads, ESS, converters, transmission_lines, power_flow indexing)
- information_exchange.proto & dynamic_operation.proto (for internal message definitions)
e. Computing Pipeline
- forecasting/ (long, mid, short-term forecasting modules)
- unit_commitment/ (input_check → formulation → solving → output_check)
- optimal_power_flow/ (similar pipeline for OPF)
- economic_dispatch/ (pipeline for economic dispatch)
f. Solvers
- solvers/ (convex_optimization, interior_point_method, mix_integer_solvers, Gurobi)
g. Logging
- log/ (detailed step-by-step logs for each stage)
h. Local EMS
- local_ems.py (manages single MG operation)
i. Static Info
- start_up/static_information.py (initial network topology, device specs)

##### Relationships & Data Flows
- On startup: start_up → configuration → static_information → initialize databases & connections.
- Data ingestion: data_management reads from SQL, web, API → writes to time-series database.
- Forecasting consumes historical data → produces forecasts.
- Unit commitment consumes forecasts + static network models → formulates MILP → solvers → UC schedules.
- Optimal power flow takes UC schedules → network model → formulates convex QCQP → solvers → set points.
- Economic dispatch refines set points across MG park → final dispatch decisions.
- local_ems applies decisions to each MG; universal_ems aggregates status.
- Throughout: modelling library provides shared data structures; configuration drives parameterization; logging captures every stage.

##### Architectural Patterns & Principles
- Pipeline pattern: sequential stages (forecast → UC → OPF → ED).
- Modular separation: clear boundaries between ingestion, modelling, computation, and orchestration.
- Abstraction via Protobuf: standard message formats for inter-module communication.
- Pluggable solvers: strategy pattern for solver choice.

## Diagram (Mermaid)

```mermaid
graph TB
    %% External Data Sources
    subgraph "External Data Sources" 
        style ExternalData fill:#D3D3D3,stroke:#000
        ExternalData["Weather & Web Data"]:::external
    end

    %% Entry & Orchestration
    subgraph "Orchestrators" 
        style Orchestrators fill:#DDA0DD,stroke:#000
        UEMS["Universal EMS Orchestrator"]:::orchestrator
        LEMS["Local EMS Orchestrator"]:::orchestrator
    end

    %% Startup Bootstrappers
    subgraph "Startup Bootstrappers"
        style Bootstrappers fill:#ADD8E6,stroke:#000
        SU["Start-up UEMS"]:::config
        SL["Start-up LEMS"]:::config
        SI["Static Information Loader"]:::static
    end

    %% Configuration Service
    subgraph "Configuration Service"
        style Config fill:#ADD8E6,stroke:#000
        CR["Configuration Root"]:::config
        CD["Configuration Database"]:::config
        CC["Configuration Convertors"]:::config
        CDF["Configuration Defaults"]:::config
    end

    %% Data Management
    subgraph "Data Ingestion & Management"
        style DataMgmt fill:#ADD8E6,stroke:#000
        DM_DB["Time-series DB"]:::database
        DMM["Database Management"]:::data
        DIC["Information Collection"]:::data
        DWS["Web Scraping"]:::data
        SQL["SQL Scripts"]:::data
    end

    %% Modelling Library
    subgraph "Modelling Library"
        style Modelling fill:#90EE90,stroke:#000
        Proto1["dynamic_operation.proto"]:::proto
        Proto2["information_exchange.proto"]:::proto
        Proto3["local_ems.proto"]:::proto
        Conv["convertors.py"]:::mod
        Gen["generators.py"]:::mod
        Loads["loads.py"]:::mod
        ESS["energy_storage_systems.py"]:::mod
        TL["transmission_lines.py"]:::mod
        subgraph "Power Flow Indexing"
            style PFIdx fill:#90EE90,stroke:#000
            PFIdxFiles["idx_*.py"]:::mod
        end
    end

    %% Compute Pipeline
    subgraph "Compute Pipeline"
        style Pipeline fill:#FFFF99,stroke:#000
        FC["Forecasting Module"]:::compute
        UC["Unit Commitment Module"]:::compute
        OPF["Optimal Power Flow Module"]:::compute
        ED["Economic Dispatch Module"]:::compute
    end

    %% Solver Pool
    subgraph "Solver Pool"
        style Solvers fill:#FF6347,stroke:#000
        CVX["convex_optimization.py"]:::solver
        IPM["interior_point_method.py"]:::solver
        MIP["mix_integer_solvers.py"]:::solver
    end

    %% Logging Service
    subgraph "Logging Service"
        style Logging fill:#D3D3C,stroke:#000
        Logs["Log Store"]:::logging
    end

    %% Connections
    ExternalData -->|ingest_data| DIC
    DWS -->|ingest_data| DIC
    DIC -->|write| DM_DB
    DMM --> DM_DB
    SQL --> DM_DB

    DM_DB -->|historical_data| FC
    FC -->|forecast_data| UC
    UC -->|uc_schedule| OPF
    OPF -->|opf_setpoints| ED
    ED -->|dispatch_decisions| LEMS
    LEMS -->|status_report| UEMS

    FC -->|invoke_solver| CVX
    UC -->|invoke_solver| MIP
    OPF -->|invoke_solver| IPM
    ED -->|invoke_solver| CVX

    SU --> UEMS
    SL --> LEMS
    SU --> CR
    SL --> CR
    SU --> SI
    SL --> SI

    UEMS --> CR
    LEMS --> CR
    UEMS --> SI
    LEMS --> SI

    CR -->|load_parameters| FC
    CR -->|load_parameters| UC
    CR -->|load_parameters| OPF
    CR -->|load_parameters| ED

    Proto1 --> FC
    Proto2 --> UC
    Proto3 --> LEMS
    Conv --> UC
    Gen --> FC
    Loads --> OPF
    ESS --> ED
    TL --> OPF
    PFIdxFiles --> OPF

    %% Logging arrows
    FC --> Logs
    UC --> Logs
    OPF --> Logs
    ED --> Logs
    DIC --> Logs
    DMM --> Logs
    CR --> Logs
    SU --> Logs
    SL --> Logs
    UEMS --> Logs
    LEMS --> Logs

    %% Click Events
    click UEMS "https://github.com/matrixeigs/energy_management_system/blob/master/universal_ems.py"
    click LEMS "https://github.com/matrixeigs/energy_management_system/blob/master/local_ems.py"
    click SU "https://github.com/matrixeigs/energy_management_system/blob/master/start_up/start_up_uems.py"
    click SL "https://github.com/matrixeigs/energy_management_system/blob/master/start_up/start_up_lems.py"
    click SI "https://github.com/matrixeigs/energy_management_system/blob/master/start_up/static_information.py"
    click CR "https://github.com/matrixeigs/energy_management_system/blob/master/configuration/configuration_root.py"
    click CD "https://github.com/matrixeigs/energy_management_system/blob/master/configuration/configuration_database.py"
    click CC "https://github.com/matrixeigs/energy_management_system/blob/master/configuration/configuration_convertors.py"
    click CDF "https://github.com/matrixeigs/energy_management_system/blob/master/configuration/configuration_default_.py"
    click DMM "https://github.com/matrixeigs/energy_management_system/blob/master/data_management/database_management.py"
    click DIC "https://github.com/matrixeigs/energy_management_system/blob/master/data_management/information_collection.py"
    click DWS "https://github.com/matrixeigs/energy_management_system/blob/master/data_management/web_scraping.py"
    click DM_DB "https://github.com/matrixeigs/energy_management_system/tree/master/data_management/"
    click SQL "https://github.com/matrixeigs/energy_management_system/tree/master/data_management/"
    click Proto1 "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/dynamic_operation.proto"
    click Proto2 "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/information_exchange.proto"
    click Proto3 "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/local_ems.proto"
    click Conv "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/convertors.py"
    click Gen "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/generators.py"
    click Loads "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/loads.py"
    click ESS "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/energy_storage_systems.py"
    click TL "https://github.com/matrixeigs/energy_management_system/blob/master/modelling/transmission_lines.py"
    click PFIdxFiles "https://github.com/matrixeigs/energy_management_system/tree/master/modelling/power_flow/"
    click FC "https://github.com/matrixeigs/energy_management_system/blob/master/forecasting/long_term_forecasting.py"
    click UC "https://github.com/matrixeigs/energy_management_system/blob/master/unit_commitment/main.py"
    click OPF "https://github.com/matrixeigs/energy_management_system/blob/master/optimal_power_flow/main.py"
    click ED "https://github.com/matrixeigs/energy_management_system/blob/master/economic_dispatch/main.py"
    click CVX "https://github.com/matrixeigs/energy_management_system/blob/master/solvers/convex_optimization.py"
    click IPM "https://github.com/matrixeigs/energy_management_system/blob/master/solvers/interior_point_method.py"
    click MIP "https://github.com/matrixeigs/energy_management_system/blob/master/solvers/mix_integer_solvers.py"
    click Logs "https://github.com/matrixeigs/energy_management_system/tree/master/log/"

    %% Styles
    classDef external fill:#D3D3D3,stroke:#000;
    classDef orchestrator fill:#DDA0DD,stroke:#000;
    classDef config fill:#ADD8E6,stroke:#000;
    classDef static fill:#90EE90,stroke:#000;
    classDef database fill:#FFC0CB,stroke:#000,shape:cylinder;
    classDef data fill:#ADD8E6,stroke:#000;
    classDef proto fill:#90EE90,stroke:#000;
    classDef mod fill:#90EE90,stroke:#000;
    classDef compute fill:#FFFF99,stroke:#000;
    classDef solver fill:#FF6347,stroke:#000,shape:cylinder;
    classDef logging fill:#D3D3D3,stroke:#000,shape:cylinder;
```
