How sayamkhatri/kidney-disease-detection-ml-application works
Project Type and Purpose
• This is an end-to-end machine learning (ML) application focused on predicting kidney disease. It covers the complete ML lifecycle—from data ingestion and preprocessing, through model training/evaluation, to deployment.
• The application is built with modularity in mind and integrates MLOps practices using tools like DVC for data version control, MLflow for experiment tracking, and Docker & GitHub Actions for deployment and CI/CD.
File Structure
• Key top-level directories:
• .dvc, dvc.yaml, params.yaml: Indicate the use of DVC for managing data and ML pipelines.
• mlruns: Contains MLflow experiment logs (tracking model runs, metrics, and parameters).
• config: Contains configuration files (static and dynamic settings).
• model: Where the serialized trained model (model.h5) is stored.
• research: Jupyter notebooks documenting data ingestion, model preparation, and experiments.
• src/mypackage: Contains the application’s core logic including subdirectories for components (data ingestion, model training/evaluation), configuration, constants, entities, pipelines which suggest a well-structured pipeline-based architecture.
• templates and app.py: Define the web front-end using Flask for serving results or providing UI.
• Other files like dockerfile and GitHub Actions workflows indicate containerization and CI/CD integration.
Components and Their Interactions
• Frontend / Deployment Layer:
• Component: Flask Web Application (app.py) and HTML templates (templates/)
• Role: Exposes the ML results and possibly user interactions.
• ML Pipeline Core:
• Data Ingestion Component (src/mypackage/components/data_ingestion.py and corresponding pipeline stage in stage_01_data_ingestion.py)
• Base Model Preparation (src/mypackage/components/prepare_base_model.py and stage_02_base_model.py)
• Model Training (src/mypackage/components/model_training.py and stage_03_model_training.py)
• Model Evaluation (src/mypackage/components/model_eval.py and stage_04_model_eval.py)
• Experiment and Model Management:
• MLflow (directory: mlruns) for logging experiments, metrics, parameter values, and run information.
• Model Storage (model/model.h5) for saving the trained model.
• Configuration and Control:
• Configurations stored in config folder and src/mypackage/config help manage environment settings, hyperparameters, and pipeline configurations.
• DVC (files: .dvc, dvc.yaml, params.yaml) used to track data versions and manage ML pipeline stages.
• Infrastructure and DevOps:
• CI/CD via GitHub Actions located in .github/workflows (automating testing/build, etc.).
• Containerization with Docker (dockerfile) ensuring the application is portable and easily deployable.
Architectural Patterns and Design Principles
• Modular Pipeline Architecture: The separation of components under src/mypackage (components, pipeline, config) underlines a clean separation of concerns for data ingestion, training, and evaluation stages.
• MLOps Best Practices: Integration of MLflow and DVC for reproducibility, experiment tracking, and managing data/model versions.
• Containerization and CI/CD: Use of Docker and GitHub Actions enable continuous integration, deployment, and consistent deployment environments.
Final Assembly
• Data and Configuration (DVC, params.yaml, config/)
• ML Pipeline (Broken into clear stages/components)
• Experiment Logging (MLflow and model storage)
• Web Service (Flask app)
• Infrastructure and Automation (Docker, GitHub Actions)