# souza-eab/mcti_qcn architecture

> Architecture diagram of the GitHub repository souza-eab/mcti_qcn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/souza-eab/mcti_qcn
- Repository: https://github.com/souza-eab/mcti_qcn

## Overview

##### Project Type and Purpose
- This is a domain-specific ETL (Extract, Transform, Load) pipeline for geospatial data, not a web or full-stack application. It ingests official GPKG (Geopackage) data from Brazil’s Fourth National Communication (QCN), processes and rectifies it, and publishes assets and interactive maps in Google Earth Engine (GEE).

##### File Structure Analysis
- Top-level
- README.md – project overview and workflow
- Scripts/ – main pipeline code, subdivided by “Step_0” and “Step_1”
- aux/ – auxiliary scripts and figures
- Scripts/Step_0
- 0_1_addColumn.r – append new “MB_C8” column correlating IBGE FITO to LULC classes
- 0_2_Rasterize.r – convert vector geopackage tables to raster tiles
- 0_3_GCP_to_GEE.py – ingest and upload raster tiles into GEE asset collections
- Scripts/Step_1
- .js scripts – assemble mosaics (joinTiles), blend biome-wide rasters, apply pixel-wise rectification
- aux/Script
- CalculeAreaBiomass.js – compute area/biomass in GEE
- JoinCsv.R – merge CSV outputs for reporting
- aux/figures – output charts comparing official vs. rectified data

##### Key Insights from README
- Two main phases:
1) Data preparation & ingestion into GEE (Step_0)
2) Spatial rectification and aggregation in GEE (Step_1)
- Outputs: GEE ImageCollections, interactive JS codeLinks, comparative figures.

a. Main Components
1. Source Data: GPKG files (per biome)
2. Local Processing Environment: R (rasterization, column addition), Python (GEE upload)
3. Google Earth Engine (External Service): asset storage, JavaScript API for mosaicking/blending/rectifying
4. Auxiliary Scripts: area/biomass calculation, CSV joining
5. Outputs: interactive GEE maps (codeLinks), static figures, CSV reports
b. Relationships & Data Flows
- GPKG → R scripts (0_1, 0_2) → raster tiles
- Raster tiles → Python script (0_3) → GEE Asset Collections
- Asset Collections → GEE JS scripts (joinTiles, blendBiomes, rectQCN) → rectified ImageCollections
- Rectified assets → auxiliary scripts (CalculeAreaBiomass.js, JoinCsv.R) → numeric summaries/CSV
- Final outputs → figures stored in aux/figures and interactive map links
c. Architectural Patterns
- Modular, step-based ETL pipeline
- Separation of concerns: data prep vs. data rectification
- External service integration (GEE as “processing + storage” microservice)
d. Technologies & Frameworks
- R (spatial packages, GDAL integration)
- Python (Earth Engine Python API)
- Google Earth Engine JavaScript API
- GitHub for code & figure hosting

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Source
    DataSource["GPKG Files (per biome)"]:::data

    %% Phase 1: Local Data Prep
    subgraph "Phase 1: Local Data Prep"
        A1["Add MB_C8 Column (R)"]:::phase1
        A2["Rasterize GPKG to Tiles (R)"]:::phase1
        A3["Upload Tiles to GEE (Python)"]:::phase1
    end

    %% Cloud Storage
    GEEAssets["GEE Asset Collections"]:::storage

    %% Phase 2: GEE Cloud Processing
    subgraph "Phase 2: Cloud Processing (GEE)"
        B1["Join Tiles & Assemble Mosaics (JS)"]:::phase2
        B2["Blend Biomes (JS)"]:::phase2
        B3["Rectification & Compartments (JS)"]:::phase2
    end

    %% Auxiliary Scripts & Outputs
    subgraph "Auxiliary & Outputs"
        C1["Calc Area/Biomass (JS)"]:::aux
        C2["Merge CSV Reports (R)"]:::aux
        Figures["Static Figures"]:::aux
        CSVReports["CSV Reports"]:::aux
        Interactive["Interactive Map Links"]:::aux
    end

    %% Documentation
    subgraph "Documentation"
        D1["README.md"]:::aux
        D2["Scripts/README.md"]:::aux
    end

    %% Data Flows
    DataSource -->|vector → raster| A1
    A1 -->|annotated data| A2
    A2 -->|tiles| A3
    A3 -->|upload| GEEAssets
    GEEAssets -->|Collections| B1
    B1 -->|mosaics| B2
    B2 -->|blended rasters| B3
    B3 -->|rectified Collections| GEEAssets
    GEEAssets -->|image collections| C1
    GEEAssets -->|image collections| C2
    C1 -->|area/biomass data| CSVReports
    C2 -->|merged CSV| CSVReports
    B3 -->|JS code outputs| Interactive
    CSVReports -->|reports| Figures

    %% Click Events
    click A1 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/Step_0/0_1_addColumn.r"
    click A2 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/Step_0/0_2_Rasterize.r"
    click A3 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/Step_0/0_3_GCP_to_GEE.py"
    click B1 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/Step_1/0_1_joinTiles_Rasterize_Amazonia.js"
    click B2 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/Step_1/0_2_blendBiomes.js"
    click B3 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/Step_1/0_3_rectQCN_eg_compartments.js"
    click C1 "https://github.com/souza-eab/mcti_qcn/blob/main/aux/Script/CalculeAreaBiomass.js"
    click C2 "https://github.com/souza-eab/mcti_qcn/blob/main/aux/Script/JoinCsv.R"
    click Figures "https://github.com/souza-eab/mcti_qcn/tree/main/aux/figures"
    click D1 "https://github.com/souza-eab/mcti_qcn/blob/main/README.md"
    click D2 "https://github.com/souza-eab/mcti_qcn/blob/main/Scripts/README.md"

    %% Styles
    classDef phase1 fill:#FFE4B5,stroke:#333,stroke-width:1px
    classDef phase2 fill:#E0FFFF,stroke:#333,stroke-width:1px
    classDef aux fill:#F0E68C,stroke:#333,stroke-width:1px
    classDef data fill:#98FB98,stroke:#333,stroke-width:1px
    classDef storage fill:#87CEFA,stroke:#333,stroke-width:1px
```
