How sa1f27/predictive-maintenance-mlops works
This repository is an end-to-end predictive maintenance ML system with a clear split between offline ML workflows and online inference/serving.
1) High-level system shape
The repo is organized around four major concerns:
1. Data preparation and model training
2. Model tracking and evaluation
3. Prediction serving and batch inference
4. Deployment/ops automation
The core architecture is:
• Raw industrial sensor data enters from Data/predictive_maintenance.csv
• The training pipeline ingests and transforms the data into reusable artifacts under artifacts/
• A model trainer fits and evaluates multiple ML classifiers
• MLflow is used for experiment tracking and likely model/version management
• The selected model is used by the prediction pipeline and exposed through the API/web app
• The entire service is containerized with Docker, and CI/CD is handled via GitHub Actions
• Deployment targets AWS ECR + ECS per the README
2) Main subsystems
A. Data ingestion and preparation
Located in src/components/:
• data_ingestion.py
• data_transformation.py
This is the offline ETL layer.
Role in architecture:
• Reads the raw CSV dataset
• Splits data into training and test sets
• Persists processed artifacts to artifacts/:
- artifacts/data.csv
- artifacts/train.csv
- artifacts/test.csv
Boundary:
This layer is the handoff point between raw data and model-ready data. It is separate from training so the same transformed datasets can be reused for evaluation and inference workflows.
B. Model training
Located in src/components/model_trainer.py
Role in architecture:
• Trains one or more classification models on the prepared data
• The README indicates the project compares multiple models such as:
- Random Forest
- Gradient Boosting
- Logistic Regression
- SVM
• Model selection is based on evaluation metrics like accuracy, precision, recall, and F1
Boundary:
This is the core ML experimentation layer. It depends on transformed data, but not on the web/API layer.
C. Pipeline orchestration
Located in src/pipeline/:
• train_pipeline.py
• model_evaluation.py
• predict_pipeline.py
• batch_prediction.py
These files define the executable workflows.
Their responsibilities are distinct:
• Training pipeline: end-to-end ingestion → transformation → training
• Model evaluation pipeline: measures model performance and likely decides whether the model is acceptable for deployment
• Prediction pipeline: handles single-record or API-driven inference
• Batch prediction pipeline: runs inference over an input file and writes prediction outputs
Boundary:
This layer is the orchestration layer connecting components to runtime entry points. It separates business workflows from implementation details.
D. MLOps / experiment tracking
Located in src/mlops/mlflow_manager.py
Role in architecture:
• Centralizes MLflow interactions
• Likely logs metrics, parameters, and model artifacts
• Supports reproducibility and model comparison
Boundary:
This is the bridge between training/evaluation code and external experiment tracking infrastructure. It keeps MLflow concerns out of model-training logic.
E. Serving layer / application runtime
Top-level app.py, plus templates/ and static/
Role in architecture:
• Serves the prediction application
• README indicates FastAPI as the API layer
• templates/home.html and templates/index.html plus static/css/style.css suggest there is also a lightweight HTML frontend, likely for manual prediction entry or landing pages
Boundary:
This is the online inference boundary. It should be treated separately from offline training pipelines because it consumes the saved model/artifacts rather than raw training data.
F. Utilities and cross-cutting concerns
Located in src/utils.py, src/logger.py, src/exception.py
Role in architecture:
• Shared helpers
• Structured logging
• Custom exception handling
These are cross-cutting support layers used throughout ingestion, transformation, training, and serving.
3) Data flow through the system
Offline training flow
1. Raw data ingestion
- Source: Data/predictive_maintenance.csv
2. Preprocessing and splitting
- Output to artifacts/
3. Model training
- Trains and compares candidate classifiers
4. Evaluation
- Metrics are computed and likely logged to MLflow
5. Artifact persistence
- Training outputs are stored for reuse by inference code
6. Visualization/output
- Evaluation plots appear under evaluation_plots/
Online prediction flow
1. A user or client sends input to the FastAPI app in app.py
2. The app likely calls the prediction pipeline
3. The pipeline loads the trained model and any preprocessing artifacts
4. The system returns a failure prediction to the caller
5. For batch jobs, predictions are written to predictions/ as CSV files
Batch inference flow
• Input data file is processed by src/pipeline/batch_prediction.py
• Output prediction CSVs are written into predictions/
• Example outputs already exist:
- predictions/batch_predictions_20250717_151026.csv
- predictions/batch_predictions_20250717_151925.csv
This is a separate path from interactive/API prediction and is useful for operational reporting or scheduled scoring jobs.
4) Important architectural boundaries
Training vs serving
This is the most important boundary in the repo.
• Training side: src/components/, src/pipeline/train_pipeline.py, model_evaluation.py
• Serving side: app.py, predict_pipeline.py, templates/static assets
The serving layer should not retrain models; it should only consume trained artifacts.
Offline artifacts vs runtime requests
The repository stores multiple artifact types:
• artifacts/ for processed datasets
• evaluation_plots/ for metrics/diagnostics
• predictions/ for batch inference outputs
These are evidence that the system is designed to persist intermediate ML outputs rather than only returning ephemeral results.
Model experimentation vs deployment
• MLflow handles the experimentation/model-registry side
• Docker/ECS handle production execution
• GitHub Actions coordinates build/test/deploy automation
This means the project has a clean separation between ML lifecycle management and infrastructure delivery.
5) Deployment and operations
Containerization
• Dockerfile and docker-compose.yml indicate the app runs in a containerized local/production-like environment.
• Docker is the packaging boundary around the API and model runtime.
CI/CD
• .github/workflows/main.yaml indicates an automated pipeline
• Based on the README, the workflow likely:
- builds the Docker image
- runs validation/tests
- pushes to AWS ECR
- deploys to AWS ECS
Cloud target
• The architecture is intended for AWS ECS as the runtime platform
• AWS ECR serves as the image registry
This implies the deployed unit is the containerized API application, not a separate model server.
6) Repository structure summary for diagramming
If you are drawing the architecture, the main boxes should be:
• Data source
- Data/predictive_maintenance.csv
• Offline ML pipeline
- ingestion
- transformation
- training
- evaluation
• Experiment tracking
- MLflow manager
• Artifacts store
- artifacts/, evaluation_plots/, predictions/
• Online inference service
- app.py
- FastAPI
- templates/static UI
• Batch inference service
- src/pipeline/batch_prediction.py
• Packaging/runtime
- Docker, Docker Compose
• CI/CD
- GitHub Actions
• Cloud deployment
- AWS ECR
- AWS ECS
7) What this repo is optimized for
This is not just a model notebook export. It is structured as a production ML application with:
• reproducible training
• tracked experiments
• persisted artifacts
• both interactive and batch inference
• containerized deployment
• automated cloud delivery
So the architecture diagram should emphasize a production ML lifecycle rather than a single request/response web app.
Main entry points: app.py API app · model_evaluation.py Evaluate · model_trainer.py Train models · train_pipeline.py Train pipeline
Components 18
Data & artifacts
Raw inputs and persisted ML outputs
- Raw data · Industrial sensor input for the pipeline
Data/predictive_maintenance.csv - Artifacts · Processed train/test splits and derived datasets
artifacts - Eval plots · Saved evaluation diagnostics and plots
evaluation_plots - Predictions · Stored batch scoring results
predictions
Offline ML
Training, evaluation, and experiment tracking
- Ingestion · Loads raw CSV and prepares initial data
src/components/data_ingestion.py - Transform · Cleans, splits, and writes model-ready datasets
src/components/data_transformation.py - Train models · Fits and compares candidate classifiers
src/components/model_trainer.py - Evaluate · Computes metrics and determines model quality
src/pipeline/model_evaluation.py - MLflow · Logs metrics, parameters, and artifacts
src/mlops/mlflow_manager.py - Train pipeline · End-to-end offline training workflow
src/pipeline/train_pipeline.py
Online inference
API and batch prediction runtime
- Predict pipeline · Loads trained artifacts for single-request inference
src/pipeline/predict_pipeline.py - Batch scoring · Runs inference over files and writes CSV outputs
src/pipeline/batch_prediction.py - API app · FastAPI service and web entrypoint
app.py - Web UI · HTML templates and static styling for manual use
templates
Delivery & runtime
Packaging and deployment automation
- Run entrypoint · Launches the project pipelines
run_pipeline.py - Docker · Builds the runtime image and local service stack
Dockerfile - Compose · Coordinates the containerized app locally
docker-compose.yml - CI workflow · Builds and deploys through GitHub Actions
.github/workflows/main.yaml
Connections 20
- Raw data → Ingestion: raw input · Reads the industrial dataset
- Ingestion → Transform: prepared data · Hands off cleaned data for splitting and processing
- Transform → Artifacts: writes · Persists train/test and derived datasets
- Artifacts → Train models: training data · Provides reusable model-ready inputs
- Train models → Evaluate: trained model · Sends candidate models for assessment
- Evaluate → Eval plots: diagnostics · Stores evaluation outputs and plots
- Train models → MLflow: metrics/artifacts · Tracks experiments and model metadata
- MLflow → Evaluate: tracking context · Supports evaluation and comparison
- Evaluate → Train pipeline: gate · Feeds the end-to-end offline workflow
- Train pipeline → Run entrypoint: pipeline entry · Executed from the repository entrypoint
- Train pipeline → API app: saved model · Produces artifacts consumed by serving
- API app → Predict pipeline: request handling · Delegates prediction logic to the inference pipeline
- Predict pipeline → Artifacts: loads preprocessing · Uses persisted training artifacts for inference
- Predict pipeline → Predictions: batch output · Writes scored records for offline jobs
- Batch scoring → Predictions: writes · Persists batch prediction CSVs
- Web UI → API app: user interaction · Front-end pages are served by the application
- Docker → API app: packages · Containerizes the API/runtime
- Compose → Docker: local run · Starts the containerized stack
- CI workflow → Docker: builds image · Automated packaging in CI
- CI workflow → API app: deploys · Delivery pipeline targets the service runtime