How selimunal/data-engineering-zoomcamp works
Data Engineering Zoomcamp is a self-paced and cohort-based educational course, not a single data-processing service. Its main learner workflow is to follow the syllabus from foundations through orchestration, warehousing, analytics, batch and streaming, then apply the material in a project. The README also documents community support, a course UI and a workshop. Source examples illustrate selected lessons; they are not a unified runtime pipeline. The graph uses README statements and the supplied source excerpts/index for its connections; other lesson contents were not inspected.
Main entry points: README.md Course syllabus · week_6_stream_processing Stream processing · producer.py Avro producer · week_2_workflow_orchestration Orchestration
Components 22
Course curriculum
- Course syllabus
README.md - Foundations
week_1_basics_n_setup - Orchestration
week_2_workflow_orchestration - Data warehouse
week_3_data_warehouse - Analytics engineering
week_4_analytics_engineering - Batch processing
week_5_batch_processing - Stream processing
week_6_stream_processing
Lesson examples
- Ingestion workflow
cohorts/2022/week_2_data_ingestion/airflow/dags_local/data_ingestion_local.py - CSV loader
cohorts/2022/week_2_data_ingestion/airflow/dags_local/ingest_script.py - Spark stream demo
cohorts/2023/week_6_stream_processing/streaming_confluent.py - Stream settings
cohorts/2023/week_6_stream_processing/settings.py - Avro producer
week_6_stream_processing/python/avro_example/producer.py - Avro ride record
week_6_stream_processing/python/avro_example/ride_record.py - Avro record key
week_6_stream_processing/python/avro_example/ride_record_key.py - JSON ride model
week_6_stream_processing/python/json_example/ride.py - Faust ride model
week_6_stream_processing/python/streams-example/faust/taxi_rides.py
Learner support
- Community support
asking-questions.md - Course UI
Course application
- Learner project
week_7_project - dbt and PipeRider workshop
cohorts/2023/workshops/piperider.md - Cohort materials
cohorts/2023
Other components
- Learner
Connections 21
- Learner → Course syllabus: follows
README.md - Course syllabus → Foundations: starts with
README.md - Course syllabus → Orchestration: continues to
README.md - Course syllabus → Data warehouse: continues to
README.md - Course syllabus → Analytics engineering: continues to
README.md - Course syllabus → Batch processing: continues to
README.md - Course syllabus → Stream processing: continues to
README.md - Orchestration → Ingestion workflow: teaches examples
README.md - Ingestion workflow → CSV loader: calls
cohorts/2022/week_2_data_ingestion/airflow/dags_local/data_ingestion_local.py - Stream processing → Spark stream demo: teaches examples
README.md - Spark stream demo → Stream settings: imports settings
cohorts/2023/week_6_stream_processing/streaming_confluent.py - Stream processing → Avro producer: teaches examples
README.md - Avro producer → Avro ride record: serializes records
week_6_stream_processing/python/avro_example/producer.py - Avro producer → Avro record key: serializes keys
week_6_stream_processing/python/avro_example/producer.py - Stream processing → JSON ride model: teaches examples
README.md - Stream processing → Faust ride model: teaches examples
README.md - Learner → Community support: seeks help
README.md - Learner → Course UI: navigates course
README.md - Learner → Cohort materials: joins cohort
README.md - Learner → Learner project: applies learning
README.md - Learner → dbt and PipeRider workshop: explores workshop
README.md