How souza-eab/mcti_qcn works
Project Type and Purpose
• This is a domain-specific ETL (Extract, Transform, Load) pipeline for geospatial data, not a web or full-stack application. It ingests official GPKG (Geopackage) data from Brazil’s Fourth National Communication (QCN), processes and rectifies it, and publishes assets and interactive maps in Google Earth Engine (GEE).
File Structure Analysis
• Top-level
• README.md – project overview and workflow
• Scripts/ – main pipeline code, subdivided by “Step_0” and “Step_1”
• aux/ – auxiliary scripts and figures
• Scripts/Step_0
• 0_1_addColumn.r – append new “MB_C8” column correlating IBGE FITO to LULC classes
• 0_2_Rasterize.r – convert vector geopackage tables to raster tiles
• 0_3_GCP_to_GEE.py – ingest and upload raster tiles into GEE asset collections
• Scripts/Step_1
• .js scripts – assemble mosaics (joinTiles), blend biome-wide rasters, apply pixel-wise rectification
• aux/Script
• CalculeAreaBiomass.js – compute area/biomass in GEE
• JoinCsv.R – merge CSV outputs for reporting
• aux/figures – output charts comparing official vs. rectified data
Key Insights from README
• Two main phases:
1) Data preparation & ingestion into GEE (Step_0)
2) Spatial rectification and aggregation in GEE (Step_1)
• Outputs: GEE ImageCollections, interactive JS codeLinks, comparative figures.
a. Main Components
1. Source Data: GPKG files (per biome)
2. Local Processing Environment: R (rasterization, column addition), Python (GEE upload)
3. Google Earth Engine (External Service): asset storage, JavaScript API for mosaicking/blending/rectifying
4. Auxiliary Scripts: area/biomass calculation, CSV joining
5. Outputs: interactive GEE maps (codeLinks), static figures, CSV reports
b. Relationships & Data Flows
• GPKG → R scripts (0_1, 0_2) → raster tiles
• Raster tiles → Python script (0_3) → GEE Asset Collections
• Asset Collections → GEE JS scripts (joinTiles, blendBiomes, rectQCN) → rectified ImageCollections
• Rectified assets → auxiliary scripts (CalculeAreaBiomass.js, JoinCsv.R) → numeric summaries/CSV
• Final outputs → figures stored in aux/figures and interactive map links
c. Architectural Patterns
• Modular, step-based ETL pipeline
• Separation of concerns: data prep vs. data rectification
• External service integration (GEE as “processing + storage” microservice)
d. Technologies & Frameworks
• R (spatial packages, GDAL integration)
• Python (Earth Engine Python API)
• Google Earth Engine JavaScript API
• GitHub for code & figure hosting