# rsaikali/linkystat architecture

> Architecture diagram of the GitHub repository rsaikali/linkystat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rsaikali/linkystat
- Repository: https://github.com/rsaikali/linkystat

## Overview

This repository is an **edge/IoT monitoring stack** deployed via **Docker Compose** (intended for a Raspberry Pi running 24/7). Its purpose is to:
- **Read electricity meter telemetry** from a **Linky** meter via a **Micro Teleinfo USB dongle** (`LINKY_USB_DEVICE`).
- **Enrich/compare with weather data** via **OpenWeather API**.
- **Persist time-series-ish measurements** into **MySQL**.

So the “system” is not a classical web app with frontend/backend; it’s a **containerized ingestion + storage + visualization** architecture running close to hardware.

From the tree:

##### Runtime / deployment orchestration
- `compose.yaml`, `compose.dev.yaml`: **defines services**, networks, volumes, device passthrough (USB), env vars.
- `Dockerfile`: likely for the Python ingestion image.
- `env/.env.sample` (README mentions `.env.example`): **runtime configuration** (device path, OpenWeather creds, DB creds, Grafana admin creds).
- `.devcontainer/*`: dev environment (not runtime, but mentionable in a “Dev tooling” side note if needed).

##### Data ingestion & integration (Python)
- `src/linky2db.py`: core **teleinfo → DB** pipeline (serial read, parse, insert).
- `src/weather.py`: **OpenWeather → DB** or enrichment logic.
- `requirements.txt`, `pyproject.toml`: Python deps/build.

##### Storage
- `files/mysql/init-script.sh`: DB initialization, schema/users, permissions.
- `scripts/migrate_to_optimized_schema.sh`: schema evolution/optimization.
- `scripts/mysql_backup.sh`, `scripts/mysql_restore.sh`, `scripts/sync_dev_db.sh`: operational lifecycle.

##### Observability / visualization
- `files/grafana/etc/grafana.ini`: Grafana config
- `files/grafana/etc/provisioning/datasources/datasource.yaml`: pre-configured **MySQL datasource**
- `files/grafana/etc/provisioning/dashboards/dashboard.yaml` + dashboards JSON: pre-provisioned dashboards

##### CI/CD & ops automation (optional layer
- `.github/workflows/main.yml` (deploy to production), `backup.yml`, `restore.yml`: automation around deploy and DB backup/restore.

This is the most important for linkystat.

##### Hardware / External
- Linky meter
- Micro Teleinfo USB dongle
- OpenWeather API (external cloud)
- User’s Browser
- (Optional) Reverse proxy (Traefik/Caddy/nginx) as an *external* recommended component (README mentions it)

##### Host
- Raspberry Pi / Linux host running Docker Engine + Docker Compose

##### Docker Compose application (logical boundary)
- **Ingestion service (Python container)** built from `Dockerfile`
- sub-label: `src/linky2db.py`
- sub-label: `src/weather.py`
- inputs: `/dev/ttyACM0` (or configured device)
- **MySQL container**
- init script: `files/mysql/init-script.sh`
- **Grafana container**
- provisioning: dashboards + datasource
- exposed port: `3000` (from README)
- persistent volume for Grafana data (if in compose)

- Linky meter → Teleinfo dongle → (USB) → Ingestion container
- Ingestion container → MySQL
- Ingestion container → OpenWeather API
- Grafana → MySQL
- Browser → Grafana

**Important: add boundaries**
- A boundary around “Docker network” (Compose network).
- A boundary around “Host OS”.
- A boundary around “External services”.

This communicates the system accurately in one page.

This is the second most useful view because the core value is “telemetry → structured DB → dashboards”.

##### Acquisition
- Serial read from USB device (Teleinfo)
##### Parsing/Validation
- parse teleinfo frames, handle sampling rate, errors
##### Enrichment
- fetch weather from OpenWeather (keyed by lat/long)
##### Persistence
- write to MySQL tables (optionally show “raw readings” vs “optimized schema” if migration script indicates new structure)
##### Query/Visualization
- Grafana dashboard panels query MySQL

- Backup script reads MySQL → writes backup artifact (local file / remote depending on workflow)
- Restore script reads artifact → writes MySQL
- Migration script alters schema

##### Relationships & interactions to call out explicitly (don’t omit these)
- **USB device passthrough into container**: this is a defining architectural constraint. Show it as a physical link from host device node to ingestion container.
- **Grafana is not the data store**: Grafana only queries MySQL; it’s provisioned via files under `files/grafana/...`.
- **MySQL initialization and user separation**:
- App credentials (`MYSQL_USER/MYSQL_PASSWORD`)
- Grafana credentials (`GRAFANA_MYSQL_USER/GRAFANA_MYSQL_PASSWORD`)
- **Port exposure**: Grafana on `3000` (public LAN), MySQL likely internal-only (show as internal to docker network unless exposed).
- **Configuration via `.env`**: show `.env` feeding Compose/services (as a config input artifact).

- **Edge deployment**: compute runs near hardware; limited dependencies.
- **Separation of concerns**:
- ingestion (Python) vs storage (MySQL) vs visualization (Grafana)
- **Infrastructure-as-code** via Docker Compose + provisioning files:
- Grafana dashboards/datasource are declarative artifacts in repo.
- **“Pull” visualization model**: Grafana pulls from MySQL; ingestion pushes to MySQL.
- **Operational automation**:

##### Shows container boundaries and which service talks to which.
3. **Shows configuration/provisioning as first-class artifacts** (Grafana dashboards/datasource, MySQL init).

##### Middle row (Host + Docker Compose boundary)
- Ingestion container (left)
- MySQL (center, with volume under it)
- Grafana (right, with provisioning docs attached)

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================
%% View A: Container / Runtime Architecture
%% =========================

subgraph "External / Hardware"
direction TB
UserBrowser["User Browser"]:::external
OpenWeather["OpenWeather API (Cloud)"]:::externalCloud
ReverseProxy["Reverse Proxy (optional)\nTraefik/Caddy/nginx"]:::externalOptional

Linky["Linky Electricity Meter"]:::hardware
Teleinfo["Micro Teleinfo USB Dongle"]:::hardware
end

subgraph "Edge Host"
direction TB
Host["Raspberry Pi / Linux Host\nDocker Engine + Docker Compose"]:::host
USBDev["USB Device Node\n(e.g., /dev/ttyACM0)"]:::hostDevice
end

subgraph "Docker Compose Application"
direction TB

ComposeSpec["Docker Compose Spec\n(compose.yaml / compose.dev.yaml)"]:::config

subgraph "Compose Network (internal)"
direction TB

Ingest["Python Ingestion Container\n(teleinfo + weather)"]:::container
MySQL["MySQL Container\n(app user + grafana user)"]:::containerDB
Grafana["Grafana Container\n(port 3000)"]:::container

MySQLVol[("MySQL Volume\n(persistent)")]:::volume
GrafanaVol[("Grafana Volume\n(persistent)")]:::volume

end

end

%% Config / provisioning artifacts
subgraph "Repo Provisioning & Config Artifacts"
direction TB
EnvFile[".env template\n(runtime configuration)"]:::config
PyDeps["Python deps/build\n(requirements.txt, pyproject.toml)"]:::config
MySQLInit["MySQL init script\n(schema/users/permissions)"]:::config
GrafanaCfg["Grafana config\n(grafana.ini)"]:::config
GrafanaDS["Grafana datasource provisioning\n(MySQL datasource.yaml)"]:::config
GrafanaDashProv["Grafana dashboard provisioning\n(dashboard.yaml)"]:::config
GrafanaDashJSON["Grafana dashboards JSON\n(full + small)"]:::config
end

%% Ops / CI
subgraph "Operations / CI Automation (dashed flows)"
direction TB
BackupScript["DB Backup Script\n(mysql_backup.sh)"]:::ops
RestoreScript["DB Restore Script\n(mysql_restore.sh)"]:::ops
MigrateScript["Schema Migration Script\n(migrate_to_optimized_schema.sh)"]:::ops
SyncDevDB["Dev DB Sync Helper\n(sync_dev_db.sh)"]:::ops

GHADeploy["GitHub Actions\nDeploy workflow"]:::ops
GHABackup["GitHub Actions\nBackup workflow"]:::ops
GHARestore["GitHub Actions\nRestore workflow"]:::ops

BackupArtifact[("Backup Artifact / Storage")]:::artifact
end

%% Runtime data flows (solid)
Linky -->|"Electricity consumption/production"| Teleinfo
Teleinfo -->|"Teleinfo frames over serial/USB"| USBDev
USBDev -->|"USB passthrough (device mount)"| Ingest

Ingest -->|"HTTP requests (weather by lat/long)"| OpenWeather
Ingest -->|"SQL write (INSERT measurements/weather)"| MySQL

Grafana -->|"SQL read (dashboards/queries)"| MySQL
UserBrowser -->|"HTTP :3000"| Grafana
UserBrowser -.->|"HTTPS (recommended)"| ReverseProxy
ReverseProxy -.->|"Proxy to :3000"| Grafana

%% Provisioning / initialization (dashed)
ComposeSpec -.->|"defines services/networks/volumes/devices"| Ingest
ComposeSpec -.->|"defines services/networks/volumes"| MySQL
ComposeSpec -.->|"defines services/networks/ports"| Grafana

EnvFile -.->|"env vars (device, creds, weather)"| ComposeSpec
PyDeps -.->|"build/install deps"| Ingest

MySQLInit -.->|"init schema/users/permissions"| MySQL

GrafanaCfg -.->|"configure Grafana"| Grafana
GrafanaDS -.->|"provision datasource"| Grafana
GrafanaDashProv -.->|"provision dashboards"| Grafana
GrafanaDashJSON -.->|"dashboard definitions"| Grafana

MySQLVol --- MySQL
GrafanaVol --- Grafana

%% Ops flows (dashed)
BackupScript -.->|"reads DB"| MySQL
BackupScript -.->|"writes dump"| BackupArtifact

RestoreScript -.->|"reads dump"| BackupArtifact
RestoreScript -.->|"writes DB"| MySQL

MigrateScript -.->|"ALTER/optimize schema"| MySQL
SyncDevDB -.->|"sync dev/prod data"| MySQL

GHADeploy -.->|"deploy automation"| ComposeSpec
GHABackup -.->|"trigger backup"| BackupScript
GHARestore -.->|"trigger restore"| RestoreScript

%% Notes (as nodes to avoid long horizontal annotations)
CredNote["MySQL credential separation:\n- App: MYSQL_USER/MYSQL_PASSWORD\n- Grafana: GRAFANA_MYSQL_USER/GRAFANA_MYSQL_PASSWORD"]:::note
CredNote -.->|"authn/authz"| MySQL

PortNote["Exposure:\n- Grafana: LAN port 3000\n- MySQL: internal-only (recommended)"]:::note
PortNote -.->|"networking"| Grafana
PortNote -.->|"networking"| MySQL

%% =========================
%% View B: Data Pipeline / Flow
%% =========================

subgraph "Data Pipeline (Ingestion → Storage → Visualization)"
direction TB

Stage1["1) Acquisition\nSerial read (Teleinfo)"]:::pipeline
Stage2["2) Parsing & Validation\nFrame decode, sampling, error handling"]:::pipeline
Stage3["3) Enrichment\nFetch weather (OpenWeather)"]:::pipeline
Stage4["4) Persistence\nWrite time-series-ish measurements to MySQL"]:::pipeline
Stage5["5) Query & Visualization\nGrafana dashboards query MySQL"]:::pipeline

Stage1 -->|"serial frames"| Stage2
Stage2 -->|"normalized readings"| Stage4
Stage2 -->|"location/time context"| Stage3
Stage3 -->|"weather metrics"| Stage4
Stage4 -->|"SQL read"| Stage5

%% Ops overlays (dashed)
Stage4 -.->|"backup"| BackupArtifact
BackupArtifact -.->|"restore"| Stage4
Stage4 -.->|"migrate schema"| Stage4
end

%% Tie pipeline to runtime components (lightweight)
Ingest -.->|"implements stages 1-4"| Stage1
OpenWeather -.->|"source for stage 3"| Stage3
MySQL -.->|"storage for stage 4"| Stage4
Grafana -.->|"implements stage 5"| Stage5

%% =========================
%% Click Events (from component_mapping)
%% =========================
click ComposeSpec "https://github.com/rsaikali/linkystat/blob/main/compose.yaml"
click ComposeSpec "https://github.com/rsaikali/linkystat/blob/main/compose.dev.yaml"
click Ingest "https://github.com/rsaikali/linkystat/tree/main/Dockerfile"
click EnvFile "https://github.com/rsaikali/linkystat/blob/main/env/.env.sample"
click Ingest "https://github.com/rsaikali/linkystat/blob/main/src/linky2db.py"
click Ingest "https://github.com/rsaikali/linkystat/blob/main/src/weather.py"
click PyDeps "https://github.com/rsaikali/linkystat/blob/main/requirements.txt"
click PyDeps "https://github.com/rsaikali/linkystat/blob/main/pyproject.toml"
click MySQLInit "https://github.com/rsaikali/linkystat/blob/main/files/mysql/init-script.sh"
click MigrateScript "https://github.com/rsaikali/linkystat/blob/main/scripts/migrate_to_optimized_schema.sh"
click BackupScript "https://github.com/rsaikali/linkystat/blob/main/scripts/mysql_backup.sh"
click RestoreScript "https://github.com/rsaikali/linkystat/blob/main/scripts/mysql_restore.sh"
click SyncDevDB "https://github.com/rsaikali/linkystat/blob/main/scripts/sync_dev_db.sh"
click GrafanaCfg "https://github.com/rsaikali/linkystat/blob/main/files/grafana/etc/grafana.ini"
click GrafanaDS "https://github.com/rsaikali/linkystat/blob/main/files/grafana/etc/provisioning/datasources/datasource.yaml"
click GrafanaDashProv "https://github.com/rsaikali/linkystat/blob/main/files/grafana/etc/provisioning/dashboards/dashboard.yaml"
click GrafanaDashJSON "https://github.com/rsaikali/linkystat/blob/main/files/grafana/etc/dashboards/grafana_dashboard.json"
click GrafanaDashJSON "https://github.com/rsaikali/linkystat/blob/main/files/grafana/etc/dashboards/grafana_dashboard_small.json"
click GHADeploy "https://github.com/rsaikali/linkystat/blob/main/.github/workflows/main.yml"
click GHABackup "https://github.com/rsaikali/linkystat/blob/main/.github/workflows/backup.yml"
click GHARestore "https://github.com/rsaikali/linkystat/blob/main/.github/workflows/restore.yml"

%% Dev tooling (optional mapping as side component)
Devcontainer["Dev Tooling\nVS Code Devcontainer"]:::dev
Devcontainer -.->|"local dev environment"| ComposeSpec
click Devcontainer "https://github.com/rsaikali/linkystat/blob/main/.devcontainer/devcontainer.json"

%% =========================
%% Styles (color is critical)
%% =========================
classDef external fill:#DFF3E3,stroke:#1B7F3A,stroke-width:1.5px,color:#0B3D1A
classDef externalCloud fill:#CFEFD7,stroke:#1B7F3A,stroke-width:2px,color:#0B3D1A
classDef externalOptional fill:#EAF7EE,stroke:#2A8C4A,stroke-dasharray:6 4,stroke-width:1.5px,color:#0B3D1A

classDef host fill:#ECEFF4,stroke:#4C566A,stroke-width:1.5px,color:#2E3440
classDef hostDevice fill:#E5E9F0,stroke:#4C566A,stroke-width:1.5px,color:#2E3440
classDef hardware fill:#E6E6E6,stroke:#616161,stroke-width:1.5px,color:#2B2B2B

classDef container fill:#D7E7FF,stroke:#2B6CB0,stroke-width:2px,color:#102A43
classDef containerDB fill:#CFE3FF,stroke:#2B6CB0,stroke-width:2px,color:#102A43
classDef volume fill:#BFD7FF,stroke:#2B6CB0,stroke-width:2px,color:#102A43

classDef config fill:#FFE6B3,stroke:#C77800,stroke-width:1.5px,color:#4A2C00
classDef ops fill:#E8D9FF,stroke:#6B46C1,stroke-width:1.5px,color:#2D1B69
classDef artifact fill:#F0E6FF,stroke:#6B46C1,stroke-width:2px,color:#2D1B69

classDef pipeline fill:#FFF4CC,stroke:#B7791F,stroke-width:1.5px,color:#4A2C00
classDef note fill:#FFF,stroke:#718096,stroke-dasharray:4 3,stroke-width:1.2px,color:#2D3748
classDef dev fill:#F7FAFC,stroke:#2C5282,stroke-dasharray:5 3,stroke-width:1.5px,color:#1A365D
```
