# brqno/olist-analytics-pipeline architecture

> Architecture diagram of the GitHub repository brqno/olist-analytics-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/brqno/olist-analytics-pipeline
- Repository: https://github.com/brqno/olist-analytics-pipeline

## Overview

This is an Olist e-commerce analytics pipeline: public CSVs are loaded into PostgreSQL, transformed into analytical dimensions and facts with dbt, and exported as per-table CSVs to S3 for business analysts. Airflow orchestrates the scheduled workflow. The README documents the stage sequence and mart capabilities; dbt model internals were not sampled, so relationships within those stages are not expanded.

Main entry points: [`airflow/dags/load_and_transformation.py`](https://github.com/brqno/olist-analytics-pipeline/blob/main/airflow/dags/load_and_transformation.py) (Airflow DAG), [`olist_dbt/models/intermediate`](https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/intermediate) (Enrichment models), [`send/marts_to_s3.py`](https://github.com/brqno/olist-analytics-pipeline/blob/main/send/marts_to_s3.py) (Mart exporter), [`load/data_load.py`](https://github.com/brqno/olist-analytics-pipeline/blob/main/load/data_load.py) (CSV loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Data ingestion"]
  node_source_csv["Olist CSVs"]
  node_loader["CSV loader<br/>[data_load.py]"]
  node_bronze[("Bronze tables")]
end

subgraph group_transform["dbt transformation"]
  node_staging["Staging models"]
  node_intermediate["Enrichment models"]
  node_region_seed["State regions seed"]
  node_dimensions[("Dimensions")]
  node_facts[("Order facts")]
end

subgraph group_orchestrate["Workflow orchestration"]
  node_airflow["Airflow DAG"]
end

subgraph group_delivery["Analytics delivery"]
  node_exporter["Mart exporter<br/>[marts_to_s3.py]"]
  node_s3[("Amazon S3")]
end

node_analyst(("Business analyst"))

node_source_csv -->|"reads CSVs"| node_loader
node_loader -->|"loads tables"| node_bronze
node_bronze -->|"transforms source"| node_staging
node_staging -->|"enriches data"| node_intermediate
node_region_seed -->|"enriches geolocation"| node_intermediate
node_intermediate -->|"builds dimensions"| node_dimensions
node_intermediate -->|"builds facts"| node_facts
node_airflow -->|"runs load"| node_loader
node_airflow -->|"runs seed"| node_region_seed
node_airflow -->|"runs dbt"| node_staging
node_airflow -->|"runs export"| node_exporter
node_dimensions -->|"reads marts"| node_exporter
node_facts -->|"reads marts"| node_exporter
node_exporter -->|"uploads CSVs"| node_s3
node_s3 -->|"provides CSVs"| node_analyst

click node_source_csv "https://github.com/brqno/olist-analytics-pipeline/tree/main/data/raw"
click node_loader "https://github.com/brqno/olist-analytics-pipeline/blob/main/load/data_load.py"
click node_staging "https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/staging"
click node_intermediate "https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/intermediate"
click node_region_seed "https://github.com/brqno/olist-analytics-pipeline/blob/main/olist_dbt/seeds/seed_regioes_estados.csv"
click node_dimensions "https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/marts"
click node_facts "https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/marts"
click node_airflow "https://github.com/brqno/olist-analytics-pipeline/blob/main/airflow/dags/load_and_transformation.py"
click node_exporter "https://github.com/brqno/olist-analytics-pipeline/blob/main/send/marts_to_s3.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_source_csv,node_loader,node_bronze toneBlue
class node_staging,node_intermediate,node_region_seed,node_dimensions,node_facts toneAmber
class node_airflow toneMint
class node_exporter,node_s3 toneRose
class node_analyst toneIndigo
```

## Components

### Data ingestion

- [Olist CSVs](https://github.com/brqno/olist-analytics-pipeline/tree/main/data/raw): `data/raw`
- [CSV loader](https://github.com/brqno/olist-analytics-pipeline/blob/main/load/data_load.py): `load/data_load.py`
- **Bronze tables**

### dbt transformation

- [Staging models](https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/staging): `olist_dbt/models/staging`
- [Enrichment models](https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/intermediate): `olist_dbt/models/intermediate`
- [State regions seed](https://github.com/brqno/olist-analytics-pipeline/blob/main/olist_dbt/seeds/seed_regioes_estados.csv): `olist_dbt/seeds/seed_regioes_estados.csv`
- [Dimensions](https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/marts): `olist_dbt/models/marts`
- [Order facts](https://github.com/brqno/olist-analytics-pipeline/tree/main/olist_dbt/models/marts): `olist_dbt/models/marts`

### Workflow orchestration

- [Airflow DAG](https://github.com/brqno/olist-analytics-pipeline/blob/main/airflow/dags/load_and_transformation.py): `airflow/dags/load_and_transformation.py`

### Analytics delivery

- [Mart exporter](https://github.com/brqno/olist-analytics-pipeline/blob/main/send/marts_to_s3.py): `send/marts_to_s3.py`
- **Amazon S3**

### Other components

- **Business analyst**

## Connections

- Olist CSVs → CSV loader: reads CSVs (evidence: `load/data_load.py`)
- CSV loader → Bronze tables: loads tables (evidence: `README.md`)
- Bronze tables → Staging models: transforms source (evidence: `README.md`)
- Staging models → Enrichment models: enriches data (evidence: `README.md`)
- State regions seed → Enrichment models: enriches geolocation (evidence: `README.md`)
- Enrichment models → Dimensions: builds dimensions (evidence: `README.md`)
- Enrichment models → Order facts: builds facts (evidence: `README.md`)
- Airflow DAG → CSV loader: runs load (evidence: `README.md`)
- Airflow DAG → State regions seed: runs seed (evidence: `README.md`)
- Airflow DAG → Staging models: runs dbt (evidence: `README.md`)
- Airflow DAG → Mart exporter: runs export (evidence: `README.md`)
- Dimensions → Mart exporter: reads marts (evidence: `send/marts_to_s3.py`)
- Order facts → Mart exporter: reads marts (evidence: `send/marts_to_s3.py`)
- Mart exporter → Amazon S3: uploads CSVs (evidence: `send/marts_to_s3.py`)
- Amazon S3 → Business analyst: provides CSVs (evidence: `README.md`)
