# adilshamim8/prices_predictor_system architecture

> Architecture diagram of the GitHub repository adilshamim8/prices_predictor_system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adilshamim8/prices_predictor_system
- Repository: https://github.com/adilshamim8/prices_predictor_system

## Overview

1. **Overall shape**
   - This repository is a **Python ML system for house-price prediction**, not a general web app.
   - Its runtime architecture has three main layers: **core ML logic (`src/`)**, **ZenML pipeline orchestration (`steps/`, `pipelines/`)**, and **consumers (`app.py`, `sample_predict.py`)**.

2. **Primary data source**
   - The only bundled raw dataset is **`data/archive.zip`**, which contains the Ames Housing data.
   - The system starts from that ZIP, extracts/loads the dataset, and then transforms it into training and inference-ready forms.

3. **Core business logic boundary: `src/`**
   - `src/` contains the reusable ML operations:
     - `ingest_data.py` for dataset loading,
     - `handle_missing_values.py` for imputation/cleaning,
     - `feature_engineering.py` for transformations like scaling, encoding, and log transforms,
     - `outlier_detection.py` for filtering outliers,
     - `data_splitter.py` for train/test partitioning,
     - `model_building.py` for constructing the sklearn model pipeline,
     - `model_evaluator.py` for regression metrics.
   - This layer is the **implementation layer**; it should be treated as the source of truth for transformations and modeling behavior.

4. **Design-patterned extensibility in `src/`**
   - The repo explicitly uses **Strategy** and **Factory** patterns.
   - The factory logic is centered in `ingest_data.py`, which chooses the correct ingestor based on file type.
   - The strategy-style modules allow alternative behaviors for missing-value handling, feature engineering, outlier detection, and evaluation without changing the calling code.
   - Architecturally, this makes the model pipeline configurable without hardcoding a single preprocessing path.

5. **Pipeline wrapper layer: `steps/`**
   - `steps/` contains **thin ZenML step wrappers** around the core `src/` functions/classes.
   - These steps are the boundary between the pure ML logic and the orchestration framework.
   - Additional deployment-oriented helpers live here too: `model_loader.py`, `prediction_service_loader.py`, `predictor.py`, and `dynamic_importer.py`.
   - In diagram terms, `steps/` is where the repo translates business logic into runnable pipeline stages.

6. **Training flow: `pipelines/training_pipeline.py`**
   - The training pipeline is the main offline lifecycle:
     - ingest data,
     - handle missing values,
     - engineer features,
     - remove outliers,
     - split into train/test,
     - build the model,
     - evaluate it.
   - It is the central path from raw data to trained model and logged metrics.
   - The README indicates **MSE and R²** are recorded during evaluation.

7. **Experiment tracking and deployment infrastructure**
   - The project uses **ZenML** to orchestrate the pipelines and **MLflow** for both experiment tracking and model serving.
   - `config.yaml` and the README’s ZenML stack instructions show that MLflow is a required integration and acts as both:
     - the **experiment tracker** for training runs,
     - the **model deployer** for serving the trained model.
   - This means the architecture separates training execution from model serving, with MLflow as the operational bridge.

8. **Deployment flow: `pipelines/deployment_pipeline.py`**
   - The deployment pipeline represents the production handoff:
     - it triggers training,
     - deploys the selected model to an **MLflow prediction server**,
     - exposes an inference endpoint.
   - The endpoint described in the README is a REST service at `/invocations`.
   - This is the system’s serving boundary: trained artifact in, HTTP inference service out.

9. **Inference consumers: `app.py` and `sample_predict.py`**
   - `app.py` is a **Streamlit UI** that collects property attributes from a user and sends them to the deployed prediction endpoint.
   - `sample_predict.py` is a small CLI client for testing the same inference service without the UI.
   - Both are downstream consumers of the deployed model; neither contains the model logic itself.

10. **Supporting analysis and documentation**
   - `analysis/` contains the EDA notebook and analysis helpers; this is exploratory, not part of the runtime pipeline.
   - `explanations/` contains standalone reference implementations of the design patterns used in the codebase.
   - These folders support understanding and experimentation, but they are not required for inference or training execution.

11. **Entry points and operational control**
   - `run_pipeline.py` starts the training pipeline.
   - `run_deployment.py` starts or stops the deployment workflow.
   - The `Makefile` is the convenience layer for common operations like install, train, deploy, app launch, and cleanup.

12. **Key architectural boundaries to draw**
   - **Raw data boundary:** `data/archive.zip`
   - **Core ML logic boundary:** `src/`
   - **Orchestration boundary:** `steps/` + `pipelines/`
   - **Tracking/serving boundary:** ZenML + MLflow
   - **User access boundary:** Streamlit UI and CLI client
   - This separation is the main architectural feature of the repository and is what should appear clearly in any diagram.

Main entry points: [`pipelines/training_pipeline.py`](https://github.com/adilshamim8/prices_predictor_system/blob/main/pipelines/training_pipeline.py) (Training pipeline), [`pipelines/deployment_pipeline.py`](https://github.com/adilshamim8/prices_predictor_system/blob/main/pipelines/deployment_pipeline.py) (Deploy pipeline), [`src/ingest_data.py`](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/ingest_data.py) (Ingest), [`src/handle_missing_values.py`](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/handle_missing_values.py) (Missing values)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_data["Data"]
  node_n_raw_data[("Ames data<br/>raw dataset<br/>[archive.zip]")]
end

subgraph group_g_core["Core ML"]
  node_n_ingest["Ingest<br/>source loader<br/>[ingest_data.py]"]
  node_n_missing["Missing values<br/>preprocess strategy"]
  node_n_features["Features<br/>feature engineering"]
  node_n_outliers["Outliers<br/>filter strategy"]
  node_n_split["Split data<br/>data partitioning<br/>[data_splitter.py]"]
  node_n_model["Build model<br/>sklearn pipeline<br/>[model_building.py]"]
  node_n_eval["Evaluate<br/>regression metrics<br/>[model_evaluator.py]"]
end

subgraph group_g_orch["Orchestration"]
  node_n_step_ingest["Ingest step<br/>zenml step"]
  node_n_step_missing["Missing step<br/>zenml step"]
  node_n_step_features["Feature step<br/>zenml step"]
  node_n_step_outliers["Outlier step<br/>zenml step"]
  node_n_step_split["Split step<br/>zenml step"]
  node_n_step_model["Model step<br/>zenml step"]
  node_n_step_eval["Eval step<br/>zenml step"]
  node_n_train_pipe["Training pipeline<br/>zenml pipeline"]
  node_n_deploy_pipe["Deploy pipeline<br/>zenml pipeline"]
end

subgraph group_g_serving["Serving"]
  node_n_zenml{{"ZenML<br/>orchestration platform"}}
  node_n_mlflow[("MLflow<br/>tracking/serving")]
  node_n_app["Streamlit app<br/>ui consumer<br/>[app.py]"]
  node_n_cli["Predict CLI<br/>client<br/>[sample_predict.py]"]
end

subgraph group_g_support["Support"]
  node_n_run_train["Run train<br/>entry point<br/>[run_pipeline.py]"]
  node_n_run_deploy["Run deploy<br/>entry point<br/>[run_deployment.py]"]
  node_n_config["Config<br/>[config.yaml]"]
  node_n_analysis["EDA<br/>analysis notebook<br/>[EDA.ipynb]"]
  node_n_patterns["Patterns<br/>reference impl"]
end

node_n_raw_data -->|"load"| node_n_ingest
node_n_ingest -->|"clean"| node_n_missing
node_n_missing -->|"transform"| node_n_features
node_n_features -->|"filter"| node_n_outliers
node_n_outliers -->|"partition"| node_n_split
node_n_split -->|"fit"| node_n_model
node_n_model -->|"score"| node_n_eval
node_n_step_ingest -->|"wraps"| node_n_ingest
node_n_step_missing -->|"wraps"| node_n_missing
node_n_step_features -->|"wraps"| node_n_features
node_n_step_outliers -->|"wraps"| node_n_outliers
node_n_step_split -->|"wraps"| node_n_split
node_n_step_model -->|"wraps"| node_n_model
node_n_step_eval -->|"wraps"| node_n_eval
node_n_train_pipe -->|"uses"| node_n_step_ingest
node_n_train_pipe -->|"uses"| node_n_step_missing
node_n_train_pipe -->|"uses"| node_n_step_features
node_n_train_pipe -->|"uses"| node_n_step_outliers
node_n_train_pipe -->|"uses"| node_n_step_split
node_n_train_pipe -->|"uses"| node_n_step_model
node_n_train_pipe -->|"uses"| node_n_step_eval
node_n_deploy_pipe -->|"triggers"| node_n_train_pipe
node_n_train_pipe -->|"runs on"| node_n_zenml
node_n_deploy_pipe -->|"runs on"| node_n_zenml
node_n_train_pipe -->|"logs to"| node_n_mlflow
node_n_deploy_pipe -->|"serves via"| node_n_mlflow
node_n_config -->|"configures"| node_n_zenml
node_n_config -->|"configures"| node_n_mlflow
node_n_run_train -->|"starts"| node_n_train_pipe
node_n_run_deploy -->|"starts"| node_n_deploy_pipe
node_n_app -->|"invokes"| node_n_mlflow
node_n_cli -->|"invokes"| node_n_mlflow
node_n_analysis -.->|"inspects"| node_n_raw_data
node_n_patterns -.->|"illustrates"| node_n_ingest

click node_n_raw_data "https://github.com/adilshamim8/prices_predictor_system/blob/main/data/archive.zip"
click node_n_ingest "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/ingest_data.py"
click node_n_missing "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/handle_missing_values.py"
click node_n_features "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/feature_engineering.py"
click node_n_outliers "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/outlier_detection.py"
click node_n_split "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/data_splitter.py"
click node_n_model "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/model_building.py"
click node_n_eval "https://github.com/adilshamim8/prices_predictor_system/blob/main/src/model_evaluator.py"
click node_n_step_ingest "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/data_ingestion_step.py"
click node_n_step_missing "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/handle_missing_values_step.py"
click node_n_step_features "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/feature_engineering_step.py"
click node_n_step_outliers "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/outlier_detection_step.py"
click node_n_step_split "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/data_splitter_step.py"
click node_n_step_model "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/model_building_step.py"
click node_n_step_eval "https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/model_evaluator_step.py"
click node_n_train_pipe "https://github.com/adilshamim8/prices_predictor_system/blob/main/pipelines/training_pipeline.py"
click node_n_deploy_pipe "https://github.com/adilshamim8/prices_predictor_system/blob/main/pipelines/deployment_pipeline.py"
click node_n_app "https://github.com/adilshamim8/prices_predictor_system/blob/main/app.py"
click node_n_cli "https://github.com/adilshamim8/prices_predictor_system/blob/main/sample_predict.py"
click node_n_run_train "https://github.com/adilshamim8/prices_predictor_system/blob/main/run_pipeline.py"
click node_n_run_deploy "https://github.com/adilshamim8/prices_predictor_system/blob/main/run_deployment.py"
click node_n_config "https://github.com/adilshamim8/prices_predictor_system/blob/main/config.yaml"
click node_n_analysis "https://github.com/adilshamim8/prices_predictor_system/blob/main/analysis/EDA.ipynb"
click node_n_patterns "https://github.com/adilshamim8/prices_predictor_system/blob/main/explanations/factory_design_pattern.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_raw_data toneBlue
class node_n_ingest,node_n_missing,node_n_features,node_n_outliers,node_n_split,node_n_model,node_n_eval toneAmber
class node_n_step_ingest,node_n_step_missing,node_n_step_features,node_n_step_outliers,node_n_step_split,node_n_step_model,node_n_step_eval,node_n_train_pipe,node_n_deploy_pipe toneMint
class node_n_zenml,node_n_mlflow,node_n_app,node_n_cli toneRose
class node_n_run_train,node_n_run_deploy,node_n_config,node_n_analysis,node_n_patterns toneIndigo
```

## Components

### Data

Raw and prepared data boundary

- [Ames data](https://github.com/adilshamim8/prices_predictor_system/blob/main/data/archive.zip): raw dataset · Bundled housing dataset used as the starting input for training · `data/archive.zip`

### Core ML

Reusable ingestion, preprocessing, modeling, and evaluation logic

- [Ingest](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/ingest_data.py): source loader · Loads data and selects the correct ingestor via factory logic · `src/ingest_data.py`
- [Missing values](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/handle_missing_values.py): preprocess strategy · Imputation and cleaning logic for absent values · `src/handle_missing_values.py`
- [Features](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/feature_engineering.py): feature engineering · Encoding, scaling, and log transforms · `src/feature_engineering.py`
- [Outliers](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/outlier_detection.py): filter strategy · Outlier detection and removal logic · `src/outlier_detection.py`
- [Split data](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/data_splitter.py): data partitioning · Train/test splitting for model development · `src/data_splitter.py`
- [Build model](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/model_building.py): sklearn pipeline · Constructs the training pipeline and estimator · `src/model_building.py`
- [Evaluate](https://github.com/adilshamim8/prices_predictor_system/blob/main/src/model_evaluator.py): regression metrics · Computes model quality metrics such as MSE and R² · `src/model_evaluator.py`

### Orchestration

ZenML pipeline and step wrappers

- [Ingest step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/data_ingestion_step.py): zenml step · Thin pipeline wrapper around data ingestion · `steps/data_ingestion_step.py`
- [Missing step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/handle_missing_values_step.py): zenml step · Pipeline wrapper for missing-value handling · `steps/handle_missing_values_step.py`
- [Feature step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/feature_engineering_step.py): zenml step · Pipeline wrapper for feature engineering · `steps/feature_engineering_step.py`
- [Outlier step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/outlier_detection_step.py): zenml step · Pipeline wrapper for outlier filtering · `steps/outlier_detection_step.py`
- [Split step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/data_splitter_step.py): zenml step · Pipeline wrapper for train/test splitting · `steps/data_splitter_step.py`
- [Model step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/model_building_step.py): zenml step · Pipeline wrapper for model construction · `steps/model_building_step.py`
- [Eval step](https://github.com/adilshamim8/prices_predictor_system/blob/main/steps/model_evaluator_step.py): zenml step · Pipeline wrapper for regression evaluation · `steps/model_evaluator_step.py`
- [Training pipeline](https://github.com/adilshamim8/prices_predictor_system/blob/main/pipelines/training_pipeline.py): zenml pipeline · Offline flow from raw data to trained and evaluated model · `pipelines/training_pipeline.py`
- [Deploy pipeline](https://github.com/adilshamim8/prices_predictor_system/blob/main/pipelines/deployment_pipeline.py): zenml pipeline · Triggers training and hands off the selected model for serving · `pipelines/deployment_pipeline.py`

### Serving

Deployment and inference consumers

- **ZenML**: orchestration platform · Pipeline orchestration layer used to run training and deployment workflows
- **MLflow**: tracking/serving · Experiment tracking and model serving backend
- [Streamlit app](https://github.com/adilshamim8/prices_predictor_system/blob/main/app.py): ui consumer · Interactive UI that sends property data to the prediction service · `app.py`
- [Predict CLI](https://github.com/adilshamim8/prices_predictor_system/blob/main/sample_predict.py): client · Command-line inference client for the deployed service · `sample_predict.py`

### Support

Entry points and exploratory/reference material

- [Run train](https://github.com/adilshamim8/prices_predictor_system/blob/main/run_pipeline.py): entry point · Starts the training pipeline · `run_pipeline.py`
- [Run deploy](https://github.com/adilshamim8/prices_predictor_system/blob/main/run_deployment.py): entry point · Starts or stops deployment workflow · `run_deployment.py`
- [Config](https://github.com/adilshamim8/prices_predictor_system/blob/main/config.yaml): stack config · ZenML stack and MLflow integration settings · `config.yaml`
- [EDA](https://github.com/adilshamim8/prices_predictor_system/blob/main/analysis/EDA.ipynb): analysis notebook · Exploratory analysis and inspection artifacts · `analysis/EDA.ipynb`
- [Patterns](https://github.com/adilshamim8/prices_predictor_system/blob/main/explanations/factory_design_pattern.py): reference impl · Standalone examples of factory and strategy patterns · `explanations/factory_design_pattern.py`

## Connections

- Ames data → Ingest: load · Raw dataset is ingested into the ML flow
- Ingest → Missing values: clean · Loaded data is cleaned and imputed
- Missing values → Features: transform · Cleaned data is transformed into model-ready features
- Features → Outliers: filter · Feature space is checked for outliers
- Outliers → Split data: partition · Filtered data is split into train and test sets
- Split data → Build model: fit · Training partitions are used to build the model
- Build model → Evaluate: score · Model predictions are evaluated on held-out data
- Ingest step → Ingest: wraps · ZenML step delegates to core ingestion logic
- Missing step → Missing values: wraps · ZenML step delegates to missing-value strategy
- Feature step → Features: wraps · ZenML step delegates to feature engineering logic
- Outlier step → Outliers: wraps · ZenML step delegates to outlier handling logic
- Split step → Split data: wraps · ZenML step delegates to splitting logic
- Model step → Build model: wraps · ZenML step delegates to model construction
- Eval step → Evaluate: wraps · ZenML step delegates to evaluation logic
- Training pipeline → Ingest step: uses · Training pipeline composes ingestion step
- Training pipeline → Missing step: uses · Training pipeline composes missing-value step
- Training pipeline → Feature step: uses · Training pipeline composes feature engineering step
- Training pipeline → Outlier step: uses · Training pipeline composes outlier step
- Training pipeline → Split step: uses · Training pipeline composes split step
- Training pipeline → Model step: uses · Training pipeline composes model-building step
- Training pipeline → Eval step: uses · Training pipeline composes evaluation step
- Deploy pipeline → Training pipeline: triggers · Deployment workflow depends on a training run
- Training pipeline → ZenML: runs on · Training pipeline is orchestrated by ZenML
- Deploy pipeline → ZenML: runs on · Deployment pipeline is orchestrated by ZenML
- Training pipeline → MLflow: logs to · Training run metrics and model artifacts go to MLflow
- Deploy pipeline → MLflow: serves via · Deployment hands off the selected model to MLflow serving
- Config → ZenML: configures · Stack configuration sets up orchestration integration
- Config → MLflow: configures · Stack configuration enables tracking and serving integration
- Run train → Training pipeline: starts · CLI entry point launches training
- Run deploy → Deploy pipeline: starts · CLI entry point launches deployment workflow
- Streamlit app → MLflow: invokes · Streamlit UI sends requests to the deployed prediction endpoint
- Predict CLI → MLflow: invokes · CLI client sends requests to the deployed prediction endpoint
- EDA → Ames data: inspects · Exploratory work examines the bundled dataset
- Patterns → Ingest: illustrates · Reference implementations explain the design patterns used in core logic