How z3yad30/de_project works
This repository is an early-stage data engineering pipeline project. Its documented target workflow is source data → Python ingestion → PostgreSQL → Kestra orchestration → dashboard. Cleaning, analytical SQL, and data-quality checks are also roadmap capabilities, not implemented components in the available tree. PostgreSQL and pgAdmin are the only documented running services. Relationships below follow the README; no source excerpts were available to verify implementation details.
Main entry points: docker-compose.yml PostgreSQL · scripts Python Ingestion · flows Kestra Flows · docs BI Dashboard
Components 9
Database Environment
- PostgreSQL
docker-compose.yml - pgAdmin
docker-compose.yml
Data Pipeline
- Python Ingestion
scripts - Data Cleaning
scripts - Kestra Flows
flows
Analytics
- Schemas and Views
sql - Quality Checks
sql - BI Dashboard
docs
Other components
- Source Data
Connections 5
- Source Data → Python Ingestion: planned input
README.md - Python Ingestion → PostgreSQL: planned load
README.md - PostgreSQL → Kestra Flows: planned orchestration
README.md - Kestra Flows → BI Dashboard: planned delivery
README.md - pgAdmin → PostgreSQL: connects to
README.md