How electricitymaps/electricitymaps-contrib works
Project type and purpose
• This is a full-stack, open-source data-pipeline and visualization platform (“Electricity Maps”) that:
• Ingests raw grid data via dozens of Python “parser” modules from official external sources (TSOs, government APIs, etc.).
• Runs processing (flow-tracing, CO₂ intensity calculation) in Python code (in /electricitymap/contrib and top-level scripts).
• Publishes a RESTful API (mockserver in Node.js for local dev, real API likely hosted by the Python backend).
• Hosts a React-based web client (/web) and an Ionic/Capacitor mobile app (/mobileapp) that consume that API.
• Contains utility scripts (/scripts) and configuration (/config) for zone definitions, emission factors, capacities.
High-level architecture components
a. External Data Sources
– Live TSO feeds, CSV/JSON/XML endpoints, public APIs, spreadsheets.
b. Parsers & Ingestion Service (Python)
– Each module in /parsers or /electricitymap/contrib/capacity_parsers fetches, normalizes, validates.
– Runs on a schedule (cron, Docker, Earthly).
c. Processing & Core Library
– Flow-tracing algorithm, CO₂ intensity parameters in /electricitymap/contrib/config and /scripts.
– Capacity update logic in capacity_update.py and scripts under /scripts.
d. Configuration & Data Store
– YAML/JSON zone definitions (/config/zones, /config/exchanges, /config/data_centers).
– Defaults (defaults.yaml), retired zones, emission factor sources.
– Final processed time-series stored in an in-memory DB or time-series DB (not shown, but implied).
e. API Layer
– Node.js mockserver (/mockserver) serves /v10/state, /v10/details, /v10/meta, /v10/gfs endpoints with static JSON for dev.
– Real backend API (Python or Node, not in this repo) serving identical endpoints.
f. Web Front-end (React + Vite)
– Lives in /web/src.
– Static assets under /web/public.
– Fetches from /api/getState, /api/getZone, /api/getMeta, /api/getWeatherData.
– Handles localization, map rendering, charts.
g. Mobile App (Ionic/Capacitor)
– Code in /mobileapp.
– Native wrappers (Android in android/, iOS in ios/), web code in src.
– Uses Capacitor to call the same REST API as the web.
h. CI/CD & DevOps
– .github/workflows for CI, code scanning, preview deployments.
– Earthfile & Dockerfiles for reproducible builds.
– Devcontainer for local VSCode.