# kharchuai/bigdata_electrivcity_forecast architecture

> Architecture diagram of the GitHub repository kharchuai/bigdata_electrivcity_forecast, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kharchuai/bigdata_electrivcity_forecast
- Repository: https://github.com/kharchuai/bigdata_electrivcity_forecast

## Overview

This repository is an early-stage electricity-forecasting pipeline: the implemented runtime covers configuration, data contracts, PostgreSQL/Iceberg initialization and infrastructure checks, and run/job tracking. The README describes ingestion, processing, feature engineering, ML, streaming and demo/benchmark work as future scaffolding, so those are not represented as working pipeline stages. Edges are limited to calls shown in excerpts or initialization relationships documented in the README; unsampled analytics wiring is omitted.

Main entry points: [`src/electricity_forecast/infrastructure.py`](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/infrastructure.py) (Infrastructure checks), [`src/electricity_forecast/cli.py`](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/cli.py) (CLI commands), [`src/electricity_forecast/metrics.py`](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/metrics.py) (Run/job tracking), [`src/electricity_forecast/config.py`](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/config.py) (Configuration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime foundation"]
  node_cli["CLI commands<br/>[cli.py]"]
  node_configuration["Configuration<br/>[config.py]"]
  node_contracts["Reading contracts<br/>[contracts.py]"]
  node_infrastructure["Infrastructure checks<br/>[infrastructure.py]"]
  node_storage["Storage primitives<br/>[storage.py]"]
end

subgraph group_operations["Run operations"]
  node_ops_metrics["Run/job tracking<br/>[metrics.py]"]
  node_sql_schema["Application schema<br/>[001_initial.sql]"]
end

subgraph group_analytics["Analytics models"]
  node_dbt_errors["Forecast error model"]
  node_dbt_metrics["Daily metrics model"]
end

node_engineer(("Engineer"))
node_postgres[("PostgreSQL")]
node_iceberg[("Iceberg on HDFS")]
node_kafka["Kafka"]

node_engineer -->|"invokes"| node_cli
node_cli -->|"loads settings"| node_configuration
node_cli -->|"runs commands"| node_infrastructure
node_cli -->|"sanitizes errors"| node_ops_metrics
node_infrastructure -->|"validates settings"| node_configuration
node_infrastructure -->|"initializes tables"| node_storage
node_storage -->|"reads settings"| node_configuration
node_storage -->|"uses keys"| node_contracts
node_infrastructure -->|"initializes schema"| node_postgres
node_infrastructure -->|"initializes tables"| node_iceberg
node_infrastructure -->|"waits for bootstrap"| node_kafka
node_ops_metrics -->|"reads settings"| node_configuration
node_ops_metrics -->|"validates timestamps"| node_contracts
node_ops_metrics -->|"records run state"| node_postgres
node_infrastructure -->|"executes SQL"| node_sql_schema

click node_cli "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/cli.py"
click node_configuration "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/config.py"
click node_contracts "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/contracts.py"
click node_infrastructure "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/infrastructure.py"
click node_storage "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/storage.py"
click node_ops_metrics "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/metrics.py"
click node_sql_schema "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/sql/001_initial.sql"
click node_dbt_errors "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/dbt/models/forecast_errors.sql"
click node_dbt_metrics "https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/dbt/models/forecast_metrics_daily.sql"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_configuration,node_contracts,node_infrastructure,node_storage toneBlue
class node_ops_metrics,node_sql_schema,node_postgres,node_iceberg toneAmber
class node_dbt_errors,node_dbt_metrics toneMint
class node_engineer,node_kafka toneIndigo
```

## Components

### Runtime foundation

- [CLI commands](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/cli.py): `src/electricity_forecast/cli.py`
- [Configuration](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/config.py): `src/electricity_forecast/config.py`
- [Reading contracts](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/contracts.py): `src/electricity_forecast/contracts.py`
- [Infrastructure checks](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/infrastructure.py): `src/electricity_forecast/infrastructure.py`
- [Storage primitives](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/storage.py): `src/electricity_forecast/storage.py`

### Run operations

- [Run/job tracking](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/src/electricity_forecast/metrics.py): `src/electricity_forecast/metrics.py`
- [Application schema](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/sql/001_initial.sql): `sql/001_initial.sql`

### Analytics models

- [Forecast error model](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/dbt/models/forecast_errors.sql): `dbt/models/forecast_errors.sql`
- [Daily metrics model](https://github.com/kharchuai/bigdata_electrivcity_forecast/blob/main/dbt/models/forecast_metrics_daily.sql): `dbt/models/forecast_metrics_daily.sql`

### Other components

- **Engineer**
- **PostgreSQL**
- **Iceberg on HDFS**
- **Kafka**

## Connections

- Engineer → CLI commands: invokes (evidence: `src/electricity_forecast/cli.py`)
- CLI commands → Configuration: loads settings (evidence: `src/electricity_forecast/cli.py`)
- CLI commands → Infrastructure checks: runs commands (evidence: `src/electricity_forecast/cli.py`)
- CLI commands → Run/job tracking: sanitizes errors (evidence: `src/electricity_forecast/cli.py`)
- Infrastructure checks → Configuration: validates settings (evidence: `src/electricity_forecast/infrastructure.py`)
- Infrastructure checks → Storage primitives: initializes tables (evidence: `src/electricity_forecast/infrastructure.py`)
- Storage primitives → Configuration: reads settings (evidence: `src/electricity_forecast/storage.py`)
- Storage primitives → Reading contracts: uses keys (evidence: `src/electricity_forecast/storage.py`)
- Infrastructure checks → PostgreSQL: initializes schema (evidence: `README.md`)
- Infrastructure checks → Iceberg on HDFS: initializes tables (evidence: `README.md`)
- Infrastructure checks → Kafka: waits for bootstrap (evidence: `README.md`)
- Run/job tracking → Configuration: reads settings (evidence: `src/electricity_forecast/metrics.py`)
- Run/job tracking → Reading contracts: validates timestamps (evidence: `src/electricity_forecast/metrics.py`)
- Run/job tracking → PostgreSQL: records run state (evidence: `src/electricity_forecast/metrics.py`)
- Infrastructure checks → Application schema: executes SQL
