# z3yad30/de_project architecture

> Architecture diagram of the GitHub repository z3yad30/de_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/z3yad30/de_project
- Repository: https://github.com/z3yad30/de_project

## Overview

This repository is an early-stage data engineering pipeline project. Its documented target workflow is source data → Python ingestion → PostgreSQL → Kestra orchestration → dashboard. Cleaning, analytical SQL, and data-quality checks are also roadmap capabilities, not implemented components in the available tree. PostgreSQL and pgAdmin are the only documented running services. Relationships below follow the README; no source excerpts were available to verify implementation details.

Main entry points: [`docker-compose.yml`](https://github.com/z3yad30/de_project/blob/main/docker-compose.yml) (PostgreSQL), [`scripts`](https://github.com/z3yad30/de_project/tree/main/scripts) (Python Ingestion), [`flows`](https://github.com/z3yad30/de_project/tree/main/flows) (Kestra Flows), [`docs`](https://github.com/z3yad30/de_project/tree/main/docs) (BI Dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_environment["Database Environment"]
  node_postgres[("PostgreSQL<br/>[docker-compose.yml]")]
  node_pgadmin["pgAdmin<br/>[docker-compose.yml]"]
end

subgraph group_pipeline["Data Pipeline"]
  node_ingestion["Python Ingestion"]
  node_cleaning["Data Cleaning"]
  node_kestra["Kestra Flows"]
end

subgraph group_analytics["Analytics"]
  node_sql["Schemas and Views"]
  node_quality["Quality Checks"]
  node_dashboard["BI Dashboard"]
end

node_source(("Source Data"))

node_source -->|"planned input"| node_ingestion
node_ingestion -->|"planned load"| node_postgres
node_postgres -->|"planned orchestration"| node_kestra
node_kestra -->|"planned delivery"| node_dashboard
node_pgadmin -->|"connects to"| node_postgres

click node_ingestion "https://github.com/z3yad30/de_project/tree/main/scripts"
click node_postgres "https://github.com/z3yad30/de_project/blob/main/docker-compose.yml"
click node_pgadmin "https://github.com/z3yad30/de_project/blob/main/docker-compose.yml"
click node_cleaning "https://github.com/z3yad30/de_project/tree/main/scripts"
click node_sql "https://github.com/z3yad30/de_project/tree/main/sql"
click node_quality "https://github.com/z3yad30/de_project/tree/main/sql"
click node_kestra "https://github.com/z3yad30/de_project/tree/main/flows"
click node_dashboard "https://github.com/z3yad30/de_project/tree/main/docs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_postgres,node_pgadmin toneBlue
class node_ingestion,node_cleaning,node_kestra toneAmber
class node_sql,node_quality,node_dashboard toneMint
class node_source toneIndigo
```

## Components

### Database Environment

- [PostgreSQL](https://github.com/z3yad30/de_project/blob/main/docker-compose.yml): `docker-compose.yml`
- [pgAdmin](https://github.com/z3yad30/de_project/blob/main/docker-compose.yml): `docker-compose.yml`

### Data Pipeline

- [Python Ingestion](https://github.com/z3yad30/de_project/tree/main/scripts): `scripts`
- [Data Cleaning](https://github.com/z3yad30/de_project/tree/main/scripts): `scripts`
- [Kestra Flows](https://github.com/z3yad30/de_project/tree/main/flows): `flows`

### Analytics

- [Schemas and Views](https://github.com/z3yad30/de_project/tree/main/sql): `sql`
- [Quality Checks](https://github.com/z3yad30/de_project/tree/main/sql): `sql`
- [BI Dashboard](https://github.com/z3yad30/de_project/tree/main/docs): `docs`

### Other components

- **Source Data**

## Connections

- Source Data → Python Ingestion: planned input (evidence: `README.md`)
- Python Ingestion → PostgreSQL: planned load (evidence: `README.md`)
- PostgreSQL → Kestra Flows: planned orchestration (evidence: `README.md`)
- Kestra Flows → BI Dashboard: planned delivery (evidence: `README.md`)
- pgAdmin → PostgreSQL: connects to (evidence: `README.md`)