# govhub-br/data-application-gov-hub architecture

> Architecture diagram of the GitHub repository govhub-br/data-application-gov-hub, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/govhub-br/data-application-gov-hub
- Repository: https://github.com/govhub-br/data-application-gov-hub

## Overview

Gov Hub BR’s runtime here is primarily an Airflow data-integration pipeline: scheduled DAGs collect public-sector and institutional data, client modules access source systems, and ingestion tasks persist records in PostgreSQL. The README describes a broader medallion flow with dbt transformation and Superset visualization; the tree includes a dbt orchestration DAG and Superset configuration, but not model definitions or dashboard implementations. Edges reflect sampled DAG code or documented capabilities; unsampled wiring is omitted.

Main entry points: [`airflow_lappis/dags/data_ingest`](https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/dags/data_ingest) (Airflow ingestion DAGs), [`airflow_lappis/plugins/cliente_postgres.py`](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_postgres.py) (PostgreSQL client), [`airflow_lappis/plugins/email_ingest_dag_factory.py`](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/email_ingest_dag_factory.py) (Email DAG factory), [`airflow_lappis/plugins`](https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/plugins) (API clients)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Data ingestion"]
  node_airflow["Airflow ingestion DAGs"]
  node_email_dag_factory["Email DAG factory"]
  node_dashboard_dag["Server dashboard DAG"]
  node_dbt_dag["dbt orchestration<br/>[cosmos_dag.py]"]
end

subgraph group_connectors["Source connectors"]
  node_api_clients["API clients"]
  node_email_client["Email client<br/>[cliente_email.py]"]
  node_sqlserver_client["SQL Server client"]
  node_helpers["Ingestion helpers"]
  node_postgres_client["PostgreSQL client"]
end

subgraph group_data["Data platform"]
  node_postgres[("PostgreSQL analytics")]
  node_dbt["dbt transformations<br/>[cosmos_dag.py]"]
end

subgraph group_insight["Analytics and serving"]
  node_superset_config["Superset configuration<br/>[superset_config.py]"]
  node_superset["Superset exploration"]
end

node_gov_manager(("Government data manager"))
node_source_apis{{"Government APIs"}}
node_email_source{{"Email reports"}}
node_sqlserver_source[("SisBolsas SQL Server")]

node_gov_manager -->|"requests reporting"| node_airflow
node_airflow -->|"invokes source clients"| node_api_clients
node_api_clients -->|"fetches records"| node_source_apis
node_airflow -->|"builds email workflows"| node_email_dag_factory
node_email_dag_factory -->|"processes attachments"| node_email_client
node_email_client -->|"retrieves reports"| node_email_source
node_airflow -->|"replicates tables"| node_sqlserver_client
node_sqlserver_client -->|"reads tables"| node_sqlserver_source
node_airflow -->|"stores ingested records"| node_postgres_client
node_email_dag_factory -->|"inserts and deduplicates"| node_postgres_client
node_airflow -->|"uses ingestion utilities"| node_helpers
node_postgres_client -->|"writes and queries"| node_postgres
node_dbt_dag -->|"orchestrates transformations"| node_dbt
node_dbt -->|"transforms data"| node_postgres
node_dashboard_dag -.->|"uses data client"| node_postgres_client
node_superset_config -->|"configures"| node_superset

click node_airflow "https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/dags/data_ingest"
click node_email_dag_factory "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/email_ingest_dag_factory.py"
click node_dashboard_dag "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/dags/dashboards/dashboard_servidores_dag.py"
click node_dbt_dag "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/dags/dbt/ipea/cosmos_dag.py"
click node_api_clients "https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/plugins"
click node_email_client "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_email.py"
click node_sqlserver_client "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_sqlserver.py"
click node_helpers "https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/helpers"
click node_postgres_client "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_postgres.py"
click node_dbt "https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/dags/dbt/ipea/cosmos_dag.py"
click node_superset_config "https://github.com/govhub-br/data-application-gov-hub/blob/main/superset/superset_config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_airflow,node_email_dag_factory,node_dashboard_dag,node_dbt_dag toneBlue
class node_api_clients,node_email_client,node_sqlserver_client,node_helpers,node_postgres_client,node_sqlserver_source toneAmber
class node_postgres,node_dbt,node_source_apis toneMint
class node_superset_config,node_superset toneRose
class node_gov_manager,node_email_source toneIndigo
```

## Components

### Data ingestion

- [Airflow ingestion DAGs](https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/dags/data_ingest): `airflow_lappis/dags/data_ingest`
- [Email DAG factory](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/email_ingest_dag_factory.py): `airflow_lappis/plugins/email_ingest_dag_factory.py`
- [Server dashboard DAG](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/dags/dashboards/dashboard_servidores_dag.py): `airflow_lappis/dags/dashboards/dashboard_servidores_dag.py`
- [dbt orchestration](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/dags/dbt/ipea/cosmos_dag.py): `airflow_lappis/dags/dbt/ipea/cosmos_dag.py`

### Source connectors

- [API clients](https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/plugins): `airflow_lappis/plugins`
- [Email client](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_email.py): `airflow_lappis/plugins/cliente_email.py`
- [SQL Server client](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_sqlserver.py): `airflow_lappis/plugins/cliente_sqlserver.py`
- [Ingestion helpers](https://github.com/govhub-br/data-application-gov-hub/tree/main/airflow_lappis/helpers): `airflow_lappis/helpers`
- [PostgreSQL client](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/plugins/cliente_postgres.py): `airflow_lappis/plugins/cliente_postgres.py`

### Data platform

- **PostgreSQL analytics**
- [dbt transformations](https://github.com/govhub-br/data-application-gov-hub/blob/main/airflow_lappis/dags/dbt/ipea/cosmos_dag.py): `airflow_lappis/dags/dbt/ipea/cosmos_dag.py`

### Analytics and serving

- [Superset configuration](https://github.com/govhub-br/data-application-gov-hub/blob/main/superset/superset_config.py): `superset/superset_config.py`
- **Superset exploration**

### Other components

- **Government data manager**
- **Government APIs**
- **Email reports**
- **SisBolsas SQL Server**

## Connections

- Government data manager → Airflow ingestion DAGs: requests reporting
- Airflow ingestion DAGs → API clients: invokes source clients
- API clients → Government APIs: fetches records
- Airflow ingestion DAGs → Email DAG factory: builds email workflows
- Email DAG factory → Email client: processes attachments
- Email client → Email reports: retrieves reports
- Airflow ingestion DAGs → SQL Server client: replicates tables
- SQL Server client → SisBolsas SQL Server: reads tables
- Airflow ingestion DAGs → PostgreSQL client: stores ingested records
- Email DAG factory → PostgreSQL client: inserts and deduplicates
- Airflow ingestion DAGs → Ingestion helpers: uses ingestion utilities
- PostgreSQL client → PostgreSQL analytics: writes and queries
- dbt orchestration → dbt transformations: orchestrates transformations
- dbt transformations → PostgreSQL analytics: transforms data
- Server dashboard DAG → PostgreSQL client: uses data client
- Superset configuration → Superset exploration: configures