How ramansh-codes/end-to-end-customer-satisfaction-pipeline works
1. What this repo is
- This is an end-to-end machine learning project for predicting e-commerce customer satisfaction scores.
- The architecture is centered on ZenML for orchestration, MLflow for experiment tracking/model deployment, and a Streamlit app for interactive inference.
2. Main architectural split
- The codebase has two major flows:
- Training / deployment pipeline: ingests data, cleans it, trains models, evaluates them, and conditionally deploys the best model.
- Inference pipeline: loads the deployed service and exposes predictions to clients such as the Streamlit app.
3. Pipeline entrypoints
- RunPipeline.py is the training-oriented entrypoint.
- RunDeployment.py is the deployment/inference entrypoint and is the main CLI for running the full ML lifecycle.
- StremLit.py is the user-facing frontend that sends feature inputs to the serving layer.
- ServerModel.py appears to provide the local model-serving alternative referenced in the README for Windows users.
4. Data ingestion boundary
- Steps/Ingest_data.py is the ingestion step that brings raw order/customer data into the pipeline.
- The dataset is described in the README as a CSV with order-level features such as payment details, product dimensions, and freight values.
- This is the first boundary where raw data becomes a pipeline artifact managed by ZenML.
5. Preprocessing / cleaning boundary
- Steps/DataPre.py performs the clean_data step.
- The actual preprocessing logic lives in src/ModelCleaning.py, which likely handles cleaning, feature preparation, and train/test splitting.
- This boundary separates raw ingested data from model-ready tabular features.
6. Training subsystem
- Steps/ModelTrain.py is the train_model step.
- src/ModelDevelopment.py contains the model training logic, and the README indicates tree-based regressors such as LightGBM and XGBoost are used.
- This is where model candidates are fit and their outputs are converted into trackable ML artifacts.
7. Evaluation subsystem
- Steps/evaluation.py runs model evaluation after training.
- src/ModelEvaluation.py defines the metrics layer, including MSE, RMSE, and R2.
- The evaluation step is the gatekeeper for deployment: model quality determines whether the pipeline continues to deployment.
8. Deployment boundary
- The deployment flow uses ZenML’s MLflow model deployer integration.
- The README shows a deployment-trigger style check using a minimum accuracy threshold before the model is promoted.
- On non-Windows systems, the inference service is started via the deployment pipeline; on Windows, the repo falls back to a local Flask-style server.
9. Inference path
- The inference pipeline is separate from training and is optimized for serving predictions.
- The README describes the sequence as dynamic_importer → prediction_service_loader → predictor.
- StremLit.py is a client of this inference path: it collects feature values and sends them to the serving endpoint for live scoring.
10. Shared ML infrastructure
- Materializer/CustomeMaterializer.py is an important infrastructure component because it lets ZenML pass custom objects such as pandas DataFrames and trained models between steps.
- Steps/config.py holds configuration classes used across the pipeline, keeping tunable model/pipeline settings out of the core logic.
- This separation keeps orchestration concerns in Steps/ and model logic in src/.
11. Operational tooling
- MLflow is the experiment and model registry layer.
- ZenML is the pipeline orchestrator and artifact manager.
- Streamlit is the presentation layer for manual testing and demos.
- The repo therefore has three distinct runtime concerns: offline training, online serving, and interactive UI.
12. Diagram-friendly mental model
- Think of the system as:
- raw CSV data in,
- ZenML-managed preprocessing,
- model training and metric evaluation,
- conditional model deployment through MLflow,
- a separate inference-serving path,
- and a Streamlit frontend consuming that service.
- The strongest architectural boundaries are between data prep, model logic, orchestration, and user-facing inference.
Main entry points: DataPre.py Clean data · ModelTrain.py Train model · evaluation.py Evaluate · RunDeployment.py RunDeployment.py
Components 19
Entry points
Pipeline and app launchers
- RunPipeline.py · Training-oriented pipeline launcher
RunPipeline.py - RunDeployment.py · Deployment and inference launcher
RunDeployment.py
Training pipeline
Ingestion, cleaning, training, and evaluation steps
- Ingest data · Raw data ingestion boundary
Steps/Ingest_data.py - Clean data · Pipeline preprocessing step
Steps/DataPre.py - Train model · Pipeline training step
Steps/ModelTrain.py - Evaluate · Pipeline evaluation step
Steps/evaluation.py - config.py · Shared pipeline configuration
Steps/config.py - ZenML · Pipeline orchestration and artifact management
Model logic
Core feature engineering, training, and metrics code
- Custom materializer · Moves DataFrames and models between steps
Materializer/CustomeMaterializer.py - ModelCleaning · Data cleaning and feature preparation
src/ModelCleaning.py - ModelDevelopment · Model fitting and candidate training
src/ModelDevelopment.py - ModelEvaluation · Regression metrics and scoring
src/ModelEvaluation.py
Serving
Deployment and inference runtime
- ServerModel.py · Local model-serving alternative
ServerModel.py - MLflow · Experiment tracking and model registry
- Prediction service · Loads deployed model and serves predictions
- Dynamic import · Imports the serving model implementation
- Service loader · Resolves the deployed prediction service
- Predictor · Runs live scoring on feature payloads
Connections 21
- RunPipeline.py → ZenML: launches · Starts the training workflow
- RunPipeline.py → Ingest data: triggers · Builds the training pipeline
- Ingest data → Clean data: passes data · Raw data becomes pipeline artifact
- Clean data → ModelCleaning: uses · Cleaning logic is implemented in src
- Clean data → Custom materializer: materializes · Transfers tabular artifacts between steps
- Clean data → Train model: feeds · Produces model-ready features
- Train model → ModelDevelopment: uses · Fits model candidates
- Train model → Custom materializer: materializes · Persists trained models and arrays
- Train model → Evaluate: hands off · Sends trained model to evaluation
- Evaluate → ModelEvaluation: uses · Computes MSE, RMSE, and R2
- Evaluate → MLflow: tracks · Records metrics and model artifacts
- Evaluate → RunDeployment.py: gates · Deployment continues only if quality is good enough
- RunDeployment.py → Prediction service: starts · Launches the inference lifecycle
- RunDeployment.py → MLflow: loads from · Uses the deployed model registry entry
- Prediction service → Dynamic import: starts with · Begins runtime loading sequence
- Dynamic import → Service loader: resolves · Locates the serving model implementation
- Service loader → Predictor: invokes · Constructs the scoring callable
- Predictor → ServerModel.py: served by · Local serving alternative for Windows users
- StremLit.py → Prediction service: calls · Frontend sends feature inputs for live scoring
- config.py → RunPipeline.py: configures · Shares tunable pipeline settings
- config.py → RunDeployment.py: configures · Shares deployment settings