How jadhavgaurav/kidney_disease_classification_cnn works
Project type and purpose
• This is a full-stack, end-to-end machine-learning application for Kidney Disease Classification.
• It provides a data versioning pipeline (DVC + GCS), experiment tracking (MLflow + DagsHub), a model‐training pipeline (VGG16-based CNN), and a Flask-based web UI, all containerized and deployed on AWS EC2 with Docker, Gunicorn, and Nginx.
File structure
• Top-level scripts:
• split_dataset.py (data prep)
• train_with_mlflow.py, mlflow_multi_run.py (model training + logging)
• app.py + static/, templates/ (Flask web app for inference)
• docker_infer/ (alternate Docker build with its own app.py, Dockerfile, templates/static)
• Hidden/config dirs:
• .dvc/ + dvc.yaml (data versioning)
• mlruns/, mlflowruns/ (local experiment logs)
• Support files: requirements.txt, EC2_Flask_App_Deployment_Guide.md
• External data: data/ (images tracked via DVC, remote GCS)
Key README insights
• Uses VGG16 transfer learning, Adam optimizer, binary_crossentropy.
• MLflow is configured to log to DagsHub remote server.
• DVC pushes data to Google Cloud Storage.
• Dockerfile (root or docker_infer) builds a container exposing port 5000.
• AWS EC2 runs Docker container, fronted by Nginx + Gunicorn.
Main components & interactions
a. Data & versioning
• Raw CT scans in data/ → tracked by DVC → pushed to GCS.
• split_dataset.py reads data/ and outputs train/test sets.
b. Training pipeline
• train_with_mlflow.py / mlflow_multi_run.py load preprocessed data → build & train VGG16-based CNN → log metrics, params, artifacts to local mlruns/ and remote DagsHub.
• Model artifacts can also be versioned via DVC if desired.
c. Model registry & storage
• MLflow remote tracking URI (DagsHub) holds model registry “KidneyDiseaseVGG16.”
d. Inference service
• Flask app (app.py) loads the registered model → exposes upload & predict endpoints → renders templates (templates/, static/).
e. Containerization & deployment
• Dockerfile builds image with Python 3.10, installs requirements, copies code, and runs “gunicorn --bind 0.0.0.0:5000 app:app.”
• AWS EC2 instance runs Docker container → Nginx proxies port 80 → port 5000 on container.
Architectural patterns & design principles
• Pipeline pattern: clear stages (data ingestion → preprocessing → training → logging → deployment).
• MVC for Flask UI:
• Model: loaded ML model
• View: HTML templates
• Controller: Flask routes in app.py
• Infrastructure-as-Code mindset for AWS (installation steps scripted in EC2 guide).
• Separation of concerns: data versioning (DVC), experiment tracking (MLflow), inference (Flask), deployment (Docker+Nginx).