How kirillkaszycki/mle-project-sprint-1-v001 works
This repository is a student project for building an apartment-price prediction pipeline. Its Airflow workflows prepare and clean apartment data in PostgreSQL; a separate DVC-oriented workflow extracts the cleaned table, fits a regression model, and evaluates it with cross-validation. The graph captures the data operations visible in the excerpts. The DVC configuration and Airflow deployment wiring are present in the tree but were not sampled, so links that depend on them are not asserted.
Main entry points: fit.py Fit model · evaluate.py Cross-validation · data.py Extract data · etl_dag.py Prepare dataset
Components 11
Airflow ETL
- Prepare dataset
part1_airflow/dags/etl_dag.py - Clean dataset
part1_airflow/dags/clean_data_dag.py
Model workflow
- Extract data
part2_dvc/scripts/data.py - Initial data CSV
part2_dvc/scripts/data.py - Fit model
part2_dvc/scripts/fit.py - Fitted model
part2_dvc/scripts/fit.py - Cross-validation
part2_dvc/scripts/evaluate.py - CV results
part2_dvc/scripts/evaluate.py
External systems
- Telegram callbacks
part1_airflow/plugins/steps/messages.py - PostgreSQL
Other components
- Airflow scheduler
Connections 16
- Airflow scheduler → Prepare dataset: schedules
- Airflow scheduler → Clean dataset: schedules
- Prepare dataset → PostgreSQL: reads and writes
part1_airflow/dags/etl_dag.py - Clean dataset → PostgreSQL: reads and writes
part1_airflow/dags/clean_data_dag.py - Prepare dataset → Telegram callbacks: registers callbacks
part1_airflow/dags/etl_dag.py - Clean dataset → Telegram callbacks: registers callbacks
part1_airflow/dags/clean_data_dag.py - Telegram callbacks → Airflow scheduler: reports status
- Extract data → PostgreSQL: reads cleaned table
part2_dvc/scripts/data.py - Extract data → Initial data CSV: writes CSV
part2_dvc/scripts/data.py - Fit model → Initial data CSV: reads CSV
part2_dvc/scripts/fit.py - Fit model → Fitted model: serializes model
part2_dvc/scripts/fit.py - Cross-validation → Initial data CSV: reads data
part2_dvc/scripts/evaluate.py - Cross-validation → Fitted model: loads model
part2_dvc/scripts/evaluate.py - Cross-validation → CV results: writes metrics
part2_dvc/scripts/evaluate.py - Extract data → Fit model: workflow order
- Fit model → Cross-validation: workflow order