# cigalesdefrance/cartes architecture

> Architecture diagram of the GitHub repository cigalesdefrance/cartes, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cigalesdefrance/cartes
- Repository: https://github.com/cigalesdefrance/cartes

## Overview

##### Project type and purpose
- This is a static, full-stack “data pipeline + static site” application for generating and hosting interactive distribution maps of French cicada species.
- Purpose: aggregate heterogeneous geolocated cicada records (CSV, geoJSON, KML) from multiple external databases (GBIF, INPN, iNaturalist, etc.), process them via ETL scripts, generate statistics and maps, then serve them as a static website with interactive maps and charts.

##### High-level components
a. External Data Sources
- GBIF, INPN, iNaturalist, ONEM, WAD, Fauna Europea, CDF, custom code lists.
b. ETL & Data-Processing Layer
- PowerShell scripts (SCRIPTS/*.ps1) for downloads, format conversions (CSV → geoJSON/KML).
- R scripts (SCRIPTS/*.R) for generating statistical outputs and static chart HTML.
c. Local Data Store
- DATA/ (canton, département, région boundaries).
- BDD/ subdirectories containing per-species CSV, geoJSON, KML.
- BDD-CODES.csv, CIGALES-CODES.csv for lookup tables.
d. Build & CI/CD
- GitHub Actions workflows (.github/workflows/*.yml) triggering data refresh, map rebuild, statistics, and site deployment.
e. Static Site & Assets
- index.html plus per-species and statistics HTML pages (STATISTIQUES/*, BDD/*/index.html).
- ASSETS/ directory with logos, style.css, OpenLayers (ol.js), custom JS (autres.js, index.js).
f. Web Client
- Browser loads static HTML, CSS, JS, fetches local geoJSON/KML for map rendering.

##### Relationships & data flow
- CI/CD triggers ETL scripts → fetch external data → writes into BDD/, DATA/.
- R scripts consume BDD/ + DATA/ → generate statistics HTML pages under STATISTIQUES/.
- Build process collates generated files + ASSETS into final static site.
- Web client requests HTML pages → loads ASSETS (JS/CSS) → fetches geoJSON/KML from BDD/ → renders interactive maps (OpenLayers).
- Users interact via browser; no dynamic backend or database server at runtime.

##### Architectural patterns & principles
- Static-site architecture (no server-side runtime code beyond build time).
- ETL pipeline for data ingestion and transformation.
- Separation of concerns: data acquisition (PowerShell), analytics (R), presentation (HTML/JS/CSS).
- CI/CD for reproducible builds and automated deployment.

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% External Data Sources
  subgraph "External Data Sources"
    classDef external fill:#D0E6FA,stroke:#4AA1F3,color:#000;
    GBIF["GBIF"]:::external
    INPN["INPN"]:::external
    INATURALIST["iNaturalist"]:::external
    CDF_SRC["CDF"]:::external
    FAUNA_EUROPEA["Fauna Europa"]:::external
    FAUNE_FRANCE["Faune France"]:::external
    GBIF_EUROPE["GBIF Europe"]:::external
    OBSERVATION["Observation"]:::external
    ONEM["ONEM"]:::external
    WAD["WAD"]:::external
  end

  %% ETL Layer
  subgraph "ETL Layer"
    classDef etlPS fill:#DFF7D8,stroke:#6BBF44,color:#000;
    classDef etlR fill:#E8F0FA,stroke:#6095D6,color:#000;
    subgraph "PowerShell ETL Scripts"
      PS1["TELECHARGEMENTS.ps1"]:::etlPS
      PS2["GEOJSON.ps1"]:::etlPS
      PS3["KML.ps1"]:::etlPS
      PS4["ALTITUDE.ps1"]:::etlPS
      PS5["CODES.ps1"]:::etlPS
      PS6["FIN.ps1"]:::etlPS
    end
    subgraph "R Analytics & Map Generation"
      R1["STATISTIQUES.R"]:::etlR
      R2["CARTES-CDF.R"]:::etlR
      R3["CARTES-EUROPE.R"]:::etlR
    end
  end

  %% Local Data Store
  subgraph "Local Data Store"
    classDef datastore fill:#FFF4C2,stroke:#E0B000,color:#000;
    direction TB
    DATA1["cantons.geojson"]:::datastore
    DATA2["departements.geojson"]:::datastore
    DATA3["regions.geojson"]:::datastore
    LUT1["BDD-CODES.csv"]:::datastore
    LUT2["CIGALES-CODES.csv"]:::datastore
    BDD_FOLDER["BDD/ (species data)"]:::datastore
  end

  %% CI/CD Pipeline
  subgraph "CI/CD (GitHub Actions)"
    classDef cicd fill:#FFE0E0,stroke:#E06666,color:#000;
    ALTITUDE_YML["ALTITUDE.yml"]:::cicd
    CARTES_YML["CARTES.yml"]:::cicd
    CODES_YML["CODES.yml"]:::cicd
    FICHIERS_YML["FICHIERS.yml"]:::cicd
    PROCESSUS_YML["PROCESSUS.yml"]:::cicd
    STATISTIQUES_YML["STATISTIQUES.yml"]:::cicd
    TELECH_YML["TELECHARGEMENTS.yml"]:::cicd
  end

  %% Build & Deployment
  subgraph "Build & Static Site Generation"
    classDef build fill:#E8D3F8,stroke:#A25FCE,color:#000;
    BUILD["Build Server"]:::build
    subgraph "Static Site Files"
      ROOT_HTML["index.html, AUTRES/index.html"]:::build
      SPECIES_HTML["Per-species index.html"]:::build
      STATS_HTML["STATISTIQUES pages"]:::build
      ASSETS_FOLDER["ASSETS/ (CSS, JS, logos)"]:::build
    end
  end

  %% Hosting
  GH_PAGES["GitHub Pages"]:::hosting
  classDef hosting fill:#F3E6FF,stroke:#A46ED3,color:#000;

  %% Web Client
  BROWSER["Web Browser"]:::client
  classDef client fill:#E0E0E0,stroke:#A0A0A0,color:#000;

  %% Connections
  GBIF -->|feeds| PS1
  INPN -->|feeds| PS1
  INATURALIST -->|feeds| PS1
  CDF_SRC -->|feeds| PS1
  FAUNA_EUROPEA -->|feeds| PS1
  FAUNE_FRANCE -->|feeds| PS1
  GBIF_EUROPE -->|feeds| PS1
  OBSERVATION -->|feeds| PS1
  ONEM -->|feeds| PS1
  WAD -->|feeds| PS1

  PS1 -->|writes| BDD_FOLDER
  PS2 -->|converts| BDD_FOLDER
  PS3 -->|converts| BDD_FOLDER
  PS4 -->|augments| BDD_FOLDER
  PS5 -->|lookups| LUT1
  PS6 -->|finalizes| BDD_FOLDER

  BDD_FOLDER -->|input| R1
  BDD_FOLDER -->|input| R2
  BDD_FOLDER -->|input| R3
  DATA1 -->|input| R1
  DATA2 -->|input| R1
  DATA3 -->|input| R1

  R1 -->|generates| STATS_HTML
  R2 -->|generates| SPECIES_HTML
  R3 -->|generates| SPECIES_HTML

  BDD_FOLDER -->|content| BUILD
  ROOT_HTML -->|content| BUILD
  SPECIES_HTML -->|content| BUILD
  STATS_HTML -->|content| BUILD
  ASSETS_FOLDER -->|assets| BUILD

  ALTITUDE_YML -->|triggers| PS1
  CARTES_YML -->|triggers| R2
  CODES_YML -->|triggers| PS5
  FICHIERS_YML -->|triggers| PS6
  PROCESSUS_YML -->|triggers| BUILD
  STATISTIQUES_YML -->|triggers| R1
  TELECH_YML -->|triggers| PS1

  BUILD -->|deploys| GH_PAGES
  GH_PAGES -->|serves| BROWSER
  BROWSER -->|requests| GH_PAGES
  BROWSER -->|fetches| ASSETS_FOLDER
  BROWSER -->|loads data| BDD_FOLDER

  %% Click Events
  click GBIF "https://github.com/cigalesdefrance/cartes/tree/main/BDD/GBIF/"
  click INPN "https://github.com/cigalesdefrance/cartes/tree/main/BDD/INPN/"
  click INATURALIST "https://github.com/cigalesdefrance/cartes/tree/main/BDD/INATURALIST/"
  click CDF_SRC "https://github.com/cigalesdefrance/cartes/tree/main/BDD/CDF/"
  click FAUNA_EUROPEA "https://github.com/cigalesdefrance/cartes/tree/main/BDD/FAUNA-EUROPEA/"
  click FAUNE_FRANCE "https://github.com/cigalesdefrance/cartes/tree/main/BDD/FAUNE-FRANCE/"
  click GBIF_EUROPE "https://github.com/cigalesdefrance/cartes/tree/main/BDD/GBIF-EUROPE/"
  click OBSERVATION "https://github.com/cigalesdefrance/cartes/tree/main/BDD/OBSERVATION/"
  click ONEM "https://github.com/cigalesdefrance/cartes/tree/main/BDD/ONEM/"
  click WAD "https://github.com/cigalesdefrance/cartes/tree/main/BDD/WAD/"
  click PS1 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/TELECHARGEMENTS.ps1"
  click PS2 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/GEOJSON.ps1"
  click PS3 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/KML.ps1"
  click PS4 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/ALTITUDE.ps1"
  click PS5 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/CODES.ps1"
  click PS6 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/FIN.ps1"
  click R1 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/STATISTIQUES.R"
  click R2 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/CARTES-CDF.R"
  click R3 "https://github.com/cigalesdefrance/cartes/blob/main/SCRIPTS/CARTES-EUROPE.R"
  click DATA1 "https://github.com/cigalesdefrance/cartes/blob/main/DATA/cantons.geojson"
  click DATA2 "https://github.com/cigalesdefrance/cartes/blob/main/DATA/departements.geojson"
  click DATA3 "https://github.com/cigalesdefrance/cartes/blob/main/DATA/regions.geojson"
  click ALTITUDE_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/ALTITUDE.yml"
  click CARTES_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/CARTES.yml"
  click CODES_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/CODES.yml"
  click FICHIERS_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/FICHIERS.yml"
  click PROCESSUS_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/PROCESSUS.yml"
  click STATISTIQUES_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/STATISTIQUES.yml"
  click TELECH_YML "https://github.com/cigalesdefrance/cartes/blob/main/.github/workflows/TELECHARGEMENTS.yml"
  click ROOT_HTML "https://github.com/cigalesdefrance/cartes/blob/main/index.html"
  click ASSETS_FOLDER "https://github.com/cigalesdefrance/cartes/tree/main/ASSETS/"
  click SPECIES_HTML "https://github.com/cigalesdefrance/cartes/blob/main/BDD/CDF/index.html"
  click STATS_HTML "https://github.com/cigalesdefrance/cartes/blob/main/STATISTIQUES/ALTITUDE/index.html"
```
