How nitin020305/ann-customer-churn-e2e-pipeline works
1. This repository is an end-to-end customer churn prediction system built around an ANN. It has two distinct runtime paths: an offline training/tuning pipeline that produces versioned artifacts, and an online Flask inference app that loads the latest trained artifacts to serve predictions.
2. The main domain flow starts with the raw dataset in notebooks/data/Churn_Modelling.csv. Training reads that CSV, validates it, transforms categorical/numeric features, trains a neural network, and writes outputs under a timestamped artifacts/ run folder. Each training run is isolated so previous runs are preserved.
3. src/churn_predictor/components/ contains the pipeline stages as separate units with clear boundaries:
- data_ingestion.py loads the CSV and creates train/test splits.
- data_validation.py checks schema and data quality.
- data_transformation.py encodes Gender, one-hot encodes Geography, and scales features using train-fitted preprocessors only.
- model_trainer.py builds and fits the ANN.
- hyperparameter_tuning.py runs GridSearchCV over ANN settings using scikeras.
- model_evaluation.py computes metrics and writes them to disk.
4. The orchestration layer lives in src/churn_predictor/pipeline/. training_pipeline.py chains the components in order and is the core of train.py. prediction_pipeline.py is separate from training and is responsible for inference-time preprocessing plus model loading.
5. The repository uses explicit typed boundaries between stages via src/churn_predictor/entity/. config_entity.py defines stage-specific configuration and artifact paths, while artifact_entity.py defines the outputs passed from one stage to the next. This keeps component dependencies explicit and reduces reliance on globals.
6. src/churn_predictor/constants.py acts as the central source of truth for magic values: file names, column names, split ratios, and ANN-related hyperparameters. That makes the training and inference logic consistent across modules and simplifies tuning.
7. Persistence and reproducibility are handled through artifacts/. The training pipeline stores preprocessors, transformed arrays, the Keras model (model.h5), TensorBoard logs, and evaluation outputs such as metrics.yaml. The prediction pipeline discovers the latest artifact set instead of hardcoding a path, so the app always targets the newest trained model.
8. Supporting infrastructure sits in src/churn_predictor/utils/main_utils.py, which provides YAML and pickle/numpy serialization helpers. Logging and error handling are centralized in src/churn_predictor/logger/ and src/churn_predictor/exception/, giving all stages a consistent operational and debugging model.
9. The runtime-facing app is app.py, backed by Flask templates in templates/index.html and styles in static/css/style.css. It does not train anything; it only collects user input, sends it through PredictionPipeline, and returns the churn result.
10. train.py and tune.py are thin entry points for the two offline workflows: full retraining and hyperparameter search. The notebooks under notebooks/ contain the original exploratory analysis and tuning experiments that were later refactored into the modular source tree.
11. Deployment is containerized with Dockerfile and docker-compose.yml. The container expects trained artifacts to exist, and compose also mounts artifacts/ so newly generated models can be picked up without rebuilding the image.
12. Tests are lightweight and focused on component behavior via tests/test_components.py. Overall, the architecture is a classic layered ML application: data ingestion/validation at the bottom, transformation and model training in the middle, artifact persistence at the boundary, and a separate inference service on top.
Main entry points: training_pipeline.py Training pipeline · data_transformation.py Transformation · prediction_pipeline.py Prediction pipeline · data_ingestion.py Ingestion
Components 20
Runtimes
Offline training/tuning and online prediction entrypoints
ML Pipeline
Core training and inference stages
- Training pipeline · Coordinates the offline end-to-end training flow
src/churn_predictor/pipeline/training_pipeline.py - Prediction pipeline · Loads latest artifacts and prepares inference-time predictions
src/churn_predictor/pipeline/prediction_pipeline.py - Ingestion · Loads CSV and creates train/test splits
src/churn_predictor/components/data_ingestion.py - Validation · Checks schema and data quality
src/churn_predictor/components/data_validation.py - Transformation · Encodes and scales features using train-fitted preprocessors
src/churn_predictor/components/data_transformation.py - Trainer · Builds and fits the ANN model
src/churn_predictor/components/model_trainer.py - Tuning · Searches ANN settings with scikeras and GridSearchCV
src/churn_predictor/components/hyperparameter_tuning.py - Evaluation · Computes metrics and writes evaluation outputs
src/churn_predictor/components/model_evaluation.py
Support
Artifacts, utilities, and shared boundaries
- Config entity · Stage-specific configuration and artifact paths
src/churn_predictor/entity/config_entity.py - Artifact entity · Structured outputs passed between stages
src/churn_predictor/entity/artifact_entity.py - Constants · File names, columns, split ratios, and hyperparameters
src/churn_predictor/constants.py - Main utils · YAML, pickle, and numpy helpers
src/churn_predictor/utils/main_utils.py - Logger · Central logging facility
src/churn_predictor/logger - Exception · Centralized error handling
src/churn_predictor/exception - artifacts/ · Versioned run outputs including model, preprocessors, logs, and metrics
artifacts - Churn CSV · Source dataset for training
notebooks/data/Churn_Modelling.csv - Component tests · Lightweight behavioral tests for pipeline components
tests/test_components.py
Connections 27
- train.py → Training pipeline: runs · Launches full training workflow
- tune.py → Tuning: runs · Launches tuning workflow
- app.py → Prediction pipeline: uses · Requests inference through the prediction pipeline
- Training pipeline → Ingestion: orders · Executes ingestion first
- Training pipeline → Validation: orders · Validates ingested data
- Training pipeline → Transformation: orders · Transforms features for training
- Training pipeline → Trainer: orders · Trains the ANN model
- Training pipeline → Evaluation: orders · Evaluates trained model
- Prediction pipeline → Transformation: reuses · Applies inference-time preprocessing
- Prediction pipeline → artifacts/: loads latest · Discovers the newest trained artifacts
- Ingestion → Churn CSV: reads · Consumes raw churn dataset
- Ingestion → Artifact entity: emits · Produces split data artifacts
- Validation → Constants: checks · Uses canonical schema and column settings
- Transformation → Constants: maps · Uses shared feature and preprocessing settings
- Trainer → Constants: configures · Uses ANN-related hyperparameters
- Tuning → Constants: searches · Uses tuning parameter space
- Evaluation → artifacts/: writes · Persists metrics and evaluation outputs
- Transformation → Main utils: serializes · Uses shared persistence helpers
- Trainer → artifacts/: writes · Stores model and run outputs
- Training pipeline → Config entity: builds · Constructs stage configuration objects
- Training pipeline → Artifact entity: passes · Threads artifacts between stages
- app.py → Logger: logs · Uses shared logging
- Training pipeline → Exception: handles · Uses centralized exception handling
- Prediction pipeline → Exception: handles · Uses centralized exception handling
- Component tests → Ingestion: covers · Checks component behavior
- Component tests → Transformation: covers · Checks component behavior
- Component tests → Trainer: covers · Checks component behavior