How govhub-br/data-application-gov-hub works
Gov Hub BR’s runtime here is primarily an Airflow data-integration pipeline: scheduled DAGs collect public-sector and institutional data, client modules access source systems, and ingestion tasks persist records in PostgreSQL. The README describes a broader medallion flow with dbt transformation and Superset visualization; the tree includes a dbt orchestration DAG and Superset configuration, but not model definitions or dashboard implementations. Edges reflect sampled DAG code or documented capabilities; unsampled wiring is omitted.
Main entry points: data_ingest Airflow ingestion DAGs · cliente_postgres.py PostgreSQL client · email_ingest_dag_factory.py Email DAG factory · plugins API clients
Components 17
Data ingestion
- Airflow ingestion DAGs
airflow_lappis/dags/data_ingest - Email DAG factory
airflow_lappis/plugins/email_ingest_dag_factory.py - Server dashboard DAG
airflow_lappis/dags/dashboards/dashboard_servidores_dag.py - dbt orchestration
airflow_lappis/dags/dbt/ipea/cosmos_dag.py
Source connectors
- API clients
airflow_lappis/plugins - Email client
airflow_lappis/plugins/cliente_email.py - SQL Server client
airflow_lappis/plugins/cliente_sqlserver.py - Ingestion helpers
airflow_lappis/helpers - PostgreSQL client
airflow_lappis/plugins/cliente_postgres.py
Data platform
- PostgreSQL analytics
- dbt transformations
airflow_lappis/dags/dbt/ipea/cosmos_dag.py
Analytics and serving
- Superset configuration
superset/superset_config.py - Superset exploration
Other components
- Government data manager
- Government APIs
- Email reports
- SisBolsas SQL Server
Connections 16
- Government data manager → Airflow ingestion DAGs: requests reporting
- Airflow ingestion DAGs → API clients: invokes source clients
- API clients → Government APIs: fetches records
- Airflow ingestion DAGs → Email DAG factory: builds email workflows
- Email DAG factory → Email client: processes attachments
- Email client → Email reports: retrieves reports
- Airflow ingestion DAGs → SQL Server client: replicates tables
- SQL Server client → SisBolsas SQL Server: reads tables
- Airflow ingestion DAGs → PostgreSQL client: stores ingested records
- Email DAG factory → PostgreSQL client: inserts and deduplicates
- Airflow ingestion DAGs → Ingestion helpers: uses ingestion utilities
- PostgreSQL client → PostgreSQL analytics: writes and queries
- dbt orchestration → dbt transformations: orchestrates transformations
- dbt transformations → PostgreSQL analytics: transforms data
- Server dashboard DAG → PostgreSQL client: uses data client
- Superset configuration → Superset exploration: configures