How adilshamim8/prices_predictor_system works
1. Overall shape
- This repository is a Python ML system for house-price prediction, not a general web app.
- Its runtime architecture has three main layers: core ML logic (src/), ZenML pipeline orchestration (steps/, pipelines/), and consumers (app.py, sample_predict.py).
2. Primary data source
- The only bundled raw dataset is data/archive.zip, which contains the Ames Housing data.
- The system starts from that ZIP, extracts/loads the dataset, and then transforms it into training and inference-ready forms.
3. Core business logic boundary: src/
- src/ contains the reusable ML operations:
- ingest_data.py for dataset loading,
- handle_missing_values.py for imputation/cleaning,
- feature_engineering.py for transformations like scaling, encoding, and log transforms,
- outlier_detection.py for filtering outliers,
- data_splitter.py for train/test partitioning,
- model_building.py for constructing the sklearn model pipeline,
- model_evaluator.py for regression metrics.
- This layer is the implementation layer; it should be treated as the source of truth for transformations and modeling behavior.
4. Design-patterned extensibility in src/
- The repo explicitly uses Strategy and Factory patterns.
- The factory logic is centered in ingest_data.py, which chooses the correct ingestor based on file type.
- The strategy-style modules allow alternative behaviors for missing-value handling, feature engineering, outlier detection, and evaluation without changing the calling code.
- Architecturally, this makes the model pipeline configurable without hardcoding a single preprocessing path.
5. Pipeline wrapper layer: steps/
- steps/ contains thin ZenML step wrappers around the core src/ functions/classes.
- These steps are the boundary between the pure ML logic and the orchestration framework.
- Additional deployment-oriented helpers live here too: model_loader.py, prediction_service_loader.py, predictor.py, and dynamic_importer.py.
- In diagram terms, steps/ is where the repo translates business logic into runnable pipeline stages.
6. Training flow: pipelines/training_pipeline.py
- The training pipeline is the main offline lifecycle:
- ingest data,
- handle missing values,
- engineer features,
- remove outliers,
- split into train/test,
- build the model,
- evaluate it.
- It is the central path from raw data to trained model and logged metrics.
- The README indicates MSE and R² are recorded during evaluation.
7. Experiment tracking and deployment infrastructure
- The project uses ZenML to orchestrate the pipelines and MLflow for both experiment tracking and model serving.
- config.yaml and the README’s ZenML stack instructions show that MLflow is a required integration and acts as both:
- the experiment tracker for training runs,
- the model deployer for serving the trained model.
- This means the architecture separates training execution from model serving, with MLflow as the operational bridge.
8. Deployment flow: pipelines/deployment_pipeline.py
- The deployment pipeline represents the production handoff:
- it triggers training,
- deploys the selected model to an MLflow prediction server,
- exposes an inference endpoint.
- The endpoint described in the README is a REST service at /invocations.
- This is the system’s serving boundary: trained artifact in, HTTP inference service out.
9. Inference consumers: app.py and sample_predict.py
- app.py is a Streamlit UI that collects property attributes from a user and sends them to the deployed prediction endpoint.
- sample_predict.py is a small CLI client for testing the same inference service without the UI.
- Both are downstream consumers of the deployed model; neither contains the model logic itself.
10. Supporting analysis and documentation
- analysis/ contains the EDA notebook and analysis helpers; this is exploratory, not part of the runtime pipeline.
- explanations/ contains standalone reference implementations of the design patterns used in the codebase.
- These folders support understanding and experimentation, but they are not required for inference or training execution.
11. Entry points and operational control
- run_pipeline.py starts the training pipeline.
- run_deployment.py starts or stops the deployment workflow.
- The Makefile is the convenience layer for common operations like install, train, deploy, app launch, and cleanup.
12. Key architectural boundaries to draw
- Raw data boundary: data/archive.zip
- Core ML logic boundary: src/
- Orchestration boundary: steps/ + pipelines/
- Tracking/serving boundary: ZenML + MLflow
- User access boundary: Streamlit UI and CLI client
- This separation is the main architectural feature of the repository and is what should appear clearly in any diagram.
Main entry points: training_pipeline.py Training pipeline · deployment_pipeline.py Deploy pipeline · ingest_data.py Ingest · handle_missing_values.py Missing values
Components 26
Data
Raw and prepared data boundary
- Ames data · Bundled housing dataset used as the starting input for training
data/archive.zip
Core ML
Reusable ingestion, preprocessing, modeling, and evaluation logic
- Ingest · Loads data and selects the correct ingestor via factory logic
src/ingest_data.py - Missing values · Imputation and cleaning logic for absent values
src/handle_missing_values.py - Features · Encoding, scaling, and log transforms
src/feature_engineering.py - Outliers · Outlier detection and removal logic
src/outlier_detection.py - Split data · Train/test splitting for model development
src/data_splitter.py - Build model · Constructs the training pipeline and estimator
src/model_building.py - Evaluate · Computes model quality metrics such as MSE and R²
src/model_evaluator.py
Orchestration
ZenML pipeline and step wrappers
- Ingest step · Thin pipeline wrapper around data ingestion
steps/data_ingestion_step.py - Missing step · Pipeline wrapper for missing-value handling
steps/handle_missing_values_step.py - Feature step · Pipeline wrapper for feature engineering
steps/feature_engineering_step.py - Outlier step · Pipeline wrapper for outlier filtering
steps/outlier_detection_step.py - Split step · Pipeline wrapper for train/test splitting
steps/data_splitter_step.py - Model step · Pipeline wrapper for model construction
steps/model_building_step.py - Eval step · Pipeline wrapper for regression evaluation
steps/model_evaluator_step.py - Training pipeline · Offline flow from raw data to trained and evaluated model
pipelines/training_pipeline.py - Deploy pipeline · Triggers training and hands off the selected model for serving
pipelines/deployment_pipeline.py
Serving
Deployment and inference consumers
- ZenML · Pipeline orchestration layer used to run training and deployment workflows
- MLflow · Experiment tracking and model serving backend
- Streamlit app · Interactive UI that sends property data to the prediction service
app.py - Predict CLI · Command-line inference client for the deployed service
sample_predict.py
Support
Entry points and exploratory/reference material
- Run train · Starts the training pipeline
run_pipeline.py - Run deploy · Starts or stops deployment workflow
run_deployment.py - Config · ZenML stack and MLflow integration settings
config.yaml - EDA · Exploratory analysis and inspection artifacts
analysis/EDA.ipynb - Patterns · Standalone examples of factory and strategy patterns
explanations/factory_design_pattern.py
Connections 34
- Ames data → Ingest: load · Raw dataset is ingested into the ML flow
- Ingest → Missing values: clean · Loaded data is cleaned and imputed
- Missing values → Features: transform · Cleaned data is transformed into model-ready features
- Features → Outliers: filter · Feature space is checked for outliers
- Outliers → Split data: partition · Filtered data is split into train and test sets
- Split data → Build model: fit · Training partitions are used to build the model
- Build model → Evaluate: score · Model predictions are evaluated on held-out data
- Ingest step → Ingest: wraps · ZenML step delegates to core ingestion logic
- Missing step → Missing values: wraps · ZenML step delegates to missing-value strategy
- Feature step → Features: wraps · ZenML step delegates to feature engineering logic
- Outlier step → Outliers: wraps · ZenML step delegates to outlier handling logic
- Split step → Split data: wraps · ZenML step delegates to splitting logic
- Model step → Build model: wraps · ZenML step delegates to model construction
- Eval step → Evaluate: wraps · ZenML step delegates to evaluation logic
- Training pipeline → Ingest step: uses · Training pipeline composes ingestion step
- Training pipeline → Missing step: uses · Training pipeline composes missing-value step
- Training pipeline → Feature step: uses · Training pipeline composes feature engineering step
- Training pipeline → Outlier step: uses · Training pipeline composes outlier step
- Training pipeline → Split step: uses · Training pipeline composes split step
- Training pipeline → Model step: uses · Training pipeline composes model-building step
- Training pipeline → Eval step: uses · Training pipeline composes evaluation step
- Deploy pipeline → Training pipeline: triggers · Deployment workflow depends on a training run
- Training pipeline → ZenML: runs on · Training pipeline is orchestrated by ZenML
- Deploy pipeline → ZenML: runs on · Deployment pipeline is orchestrated by ZenML
- Training pipeline → MLflow: logs to · Training run metrics and model artifacts go to MLflow
- Deploy pipeline → MLflow: serves via · Deployment hands off the selected model to MLflow serving
- Config → ZenML: configures · Stack configuration sets up orchestration integration
- Config → MLflow: configures · Stack configuration enables tracking and serving integration
- Run train → Training pipeline: starts · CLI entry point launches training
- Run deploy → Deploy pipeline: starts · CLI entry point launches deployment workflow
- Streamlit app → MLflow: invokes · Streamlit UI sends requests to the deployed prediction endpoint
- Predict CLI → MLflow: invokes · CLI client sends requests to the deployed prediction endpoint
- EDA → Ames data: inspects · Exploratory work examines the bundled dataset
- Patterns → Ingest: illustrates · Reference implementations explain the design patterns used in core logic