# gabriel-goncalves1122/mapeamento-risco-queimadas-inpe architecture

> Architecture diagram of the GitHub repository gabriel-goncalves1122/mapeamento-risco-queimadas-inpe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe
- Repository: https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe

## Overview

CCO DataOps (SIRDA) monitors wildfire impacts on critical infrastructure through a Bronze–Silver–Gold data platform. Its workflow acquires official data, builds cleaned spatial dimensions, enriches and clusters fire observations, then publishes fact streams for OLAP analysis and geospatial dashboards. Dagster coordinates documented pipeline assets; the repository also includes Druid lookup registration and network-centrality analysis. The README describes broader integrations and stages than the excerpts expose, so those links are cited to the README rather than inferred from unsampled code.

Main entry points: [`src/orchestration/dagster/repo.py`](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/orchestration/dagster/repo.py) (Dagster assets), [`src/silver/streaming_processing/processador.py`](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/streaming_processing/processador.py) (Focus processor), [`src/silver/etl`](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/etl) (Dimension ETLs), [`src/gold/streaming/produtor_eventos.py`](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/produtor_eventos.py) (Event producer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core and acquisition"]
  node_contracts["Pipeline contracts<br/>[contracts.py]"]
  node_settings["Pipeline settings<br/>[settings.py]"]
  node_aneel["ANEEL extractor<br/>[ingestao_aneel.py]"]
  node_inpe["INPE extractor"]
end

subgraph group_silver["Silver processing"]
  node_dimensions["Dimension ETLs"]
  node_hospitals["Hospital ETL"]
  node_silverutils["Silver utilities"]
  node_spatial["Spatial engine"]
  node_focus["Focus processor<br/>[processador.py]"]
  node_calculators["Focus calculators<br/>[calculators.py]"]
  node_lookup["Druid lookups"]
end

subgraph group_gold["Gold analytics"]
  node_events["Fire event clustering<br/>[eventizador.py]"]
  node_geometry["Event geometry"]
  node_network["Grid centrality<br/>[network_builder.py]"]
  node_focusproducer["Focus producer<br/>[produtor_focos.py]"]
  node_eventproducer["Event producer"]
  node_kafka["Kafka client<br/>[kafka_client.py]"]
  node_audit["Streaming audit"]
end

subgraph group_ops["Orchestration and OLAP"]
  node_dagster["Dagster assets<br/>[repo.py]"]
  node_druid[("Druid OLAP")]
  node_superset["Superset dashboards"]
end

node_official(("Official data sources"))
node_kafkaext["Kafka broker"]

node_official -->|"provides data"| node_aneel
node_official -->|"provides data"| node_inpe
node_aneel -->|"supplies raw layers"| node_dimensions
node_dimensions -->|"uses helpers"| node_silverutils
node_hospitals -->|"uses helpers"| node_silverutils
node_hospitals -->|"builds dimension"| node_dimensions
node_focus -->|"calls calculations"| node_calculators
node_focus -->|"queries threats"| node_spatial
node_focus -->|"cleans values"| node_silverutils
node_events -->|"builds geometry"| node_geometry
node_events -.->|"reuses enrichment"| node_focus
node_network -.->|"analyzes grid"| node_dimensions
node_focusproducer -.->|"publishes through"| node_kafka
node_eventproducer -->|"clusters observations"| node_events
node_eventproducer -->|"processes observations"| node_focus
node_eventproducer -->|"publishes through"| node_kafka
node_eventproducer -->|"records activity"| node_audit
node_kafka -->|"sends messages"| node_kafkaext
node_kafkaext -->|"feeds OLAP"| node_druid
node_lookup -->|"registers lookups"| node_druid
node_druid -->|"supplies analytics"| node_superset
node_dagster -->|"executes asset"| node_aneel
node_dagster -->|"executes asset"| node_inpe
node_dagster -->|"orchestrates assets"| node_dimensions
node_dagster -->|"executes asset"| node_network
node_dagster -->|"executes asset"| node_events
node_settings -->|"configures client"| node_kafka
node_contracts -->|"defines step contract"| node_dagster

click node_contracts "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/core/contracts/contracts.py"
click node_settings "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/core/config/settings.py"
click node_aneel "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/bronze/extractors/ingestao_aneel.py"
click node_inpe "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/bronze/extractors/inpe_queimadas_extractor.py"
click node_dimensions "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/etl"
click node_hospitals "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/etl/etl_dim_hospitais.py"
click node_silverutils "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/utils"
click node_spatial "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/spatial"
click node_focus "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/streaming_processing/processador.py"
click node_calculators "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/streaming_processing/processors/calculators.py"
click node_lookup "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/lookups/druid_lookup_registrar.py"
click node_events "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/science/dbscan/eventizador.py"
click node_geometry "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/science/dbscan/event_geometry_builder.py"
click node_network "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/science/networkx/network_builder.py"
click node_focusproducer "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/produtor_focos.py"
click node_eventproducer "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/produtor_eventos.py"
click node_kafka "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/kafka_client.py"
click node_audit "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/streaming_audit_logger.py"
click node_dagster "https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/orchestration/dagster/repo.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_contracts,node_settings,node_aneel,node_inpe toneBlue
class node_dimensions,node_hospitals,node_silverutils,node_spatial,node_focus,node_calculators,node_lookup toneAmber
class node_events,node_geometry,node_network,node_focusproducer,node_eventproducer,node_kafka,node_audit toneMint
class node_dagster,node_druid,node_superset toneRose
class node_official,node_kafkaext toneIndigo
```

## Components

### Core and acquisition

- [Pipeline contracts](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/core/contracts/contracts.py): `src/core/contracts/contracts.py`
- [Pipeline settings](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/core/config/settings.py): `src/core/config/settings.py`
- [ANEEL extractor](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/bronze/extractors/ingestao_aneel.py): `src/bronze/extractors/ingestao_aneel.py`
- [INPE extractor](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/bronze/extractors/inpe_queimadas_extractor.py): `src/bronze/extractors/inpe_queimadas_extractor.py`

### Silver processing

- [Dimension ETLs](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/etl): `src/silver/etl`
- [Hospital ETL](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/etl/etl_dim_hospitais.py): `src/silver/etl/etl_dim_hospitais.py`
- [Silver utilities](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/utils): `src/silver/utils`
- [Spatial engine](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/tree/main/src/silver/spatial): `src/silver/spatial`
- [Focus processor](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/streaming_processing/processador.py): `src/silver/streaming_processing/processador.py`
- [Focus calculators](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/streaming_processing/processors/calculators.py): `src/silver/streaming_processing/processors/calculators.py`
- [Druid lookups](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/silver/lookups/druid_lookup_registrar.py): `src/silver/lookups/druid_lookup_registrar.py`

### Gold analytics

- [Fire event clustering](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/science/dbscan/eventizador.py): `src/gold/science/dbscan/eventizador.py`
- [Event geometry](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/science/dbscan/event_geometry_builder.py): `src/gold/science/dbscan/event_geometry_builder.py`
- [Grid centrality](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/science/networkx/network_builder.py): `src/gold/science/networkx/network_builder.py`
- [Focus producer](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/produtor_focos.py): `src/gold/streaming/produtor_focos.py`
- [Event producer](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/produtor_eventos.py): `src/gold/streaming/produtor_eventos.py`
- [Kafka client](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/kafka_client.py): `src/gold/streaming/kafka_client.py`
- [Streaming audit](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/gold/streaming/streaming_audit_logger.py): `src/gold/streaming/streaming_audit_logger.py`

### Orchestration and OLAP

- [Dagster assets](https://github.com/gabriel-goncalves1122/mapeamento-risco-queimadas-inpe/blob/main/src/orchestration/dagster/repo.py): `src/orchestration/dagster/repo.py`
- **Druid OLAP**
- **Superset dashboards**

### Other components

- **Official data sources**
- **Kafka broker**

## Connections

- Official data sources → ANEEL extractor: provides data (evidence: `src/bronze/extractors/ingestao_aneel.py`)
- Official data sources → INPE extractor: provides data (evidence: `README.md`)
- ANEEL extractor → Dimension ETLs: supplies raw layers (evidence: `README.md`)
- Dimension ETLs → Silver utilities: uses helpers (evidence: `src/silver/etl/etl_dim_hospitais.py`)
- Hospital ETL → Silver utilities: uses helpers (evidence: `src/silver/etl/etl_dim_hospitais.py`)
- Hospital ETL → Dimension ETLs: builds dimension (evidence: `src/silver/etl/etl_dim_hospitais.py`)
- Focus processor → Focus calculators: calls calculations (evidence: `src/silver/streaming_processing/processador.py`)
- Focus processor → Spatial engine: queries threats (evidence: `src/silver/streaming_processing/processador.py`)
- Focus processor → Silver utilities: cleans values (evidence: `src/silver/streaming_processing/processador.py`)
- Fire event clustering → Event geometry: builds geometry (evidence: `src/gold/science/dbscan/eventizador.py`)
- Fire event clustering → Focus processor: reuses enrichment
- Grid centrality → Dimension ETLs: analyzes grid
- Focus producer → Kafka client: publishes through
- Event producer → Fire event clustering: clusters observations (evidence: `src/gold/streaming/produtor_eventos.py`)
- Event producer → Focus processor: processes observations (evidence: `src/gold/streaming/produtor_eventos.py`)
- Event producer → Kafka client: publishes through (evidence: `src/gold/streaming/produtor_eventos.py`)
- Event producer → Streaming audit: records activity (evidence: `src/gold/streaming/produtor_eventos.py`)
- Kafka client → Kafka broker: sends messages (evidence: `src/gold/streaming/kafka_client.py`)
- Kafka broker → Druid OLAP: feeds OLAP (evidence: `README.md`)
- Druid lookups → Druid OLAP: registers lookups (evidence: `README.md`)
- Druid OLAP → Superset dashboards: supplies analytics (evidence: `README.md`)
- Dagster assets → ANEEL extractor: executes asset (evidence: `src/orchestration/dagster/repo.py`)
- Dagster assets → INPE extractor: executes asset (evidence: `src/orchestration/dagster/repo.py`)
- Dagster assets → Dimension ETLs: orchestrates assets (evidence: `src/orchestration/dagster/repo.py`)
- Dagster assets → Grid centrality: executes asset (evidence: `src/orchestration/dagster/repo.py`)
- Dagster assets → Fire event clustering: executes asset (evidence: `src/orchestration/dagster/repo.py`)
- Pipeline settings → Kafka client: configures client (evidence: `src/gold/streaming/kafka_client.py`)
- Pipeline contracts → Dagster assets: defines step contract (evidence: `src/orchestration/dagster/repo.py`)
