How anvayb/beyondthetap works
Project type and purpose
• This is a data-science / machine-learning analysis project (not a client–server or microservice app).
• Its goal is to ingest water-quality data, perform exploratory data analysis (EDA), preprocess and engineer features, run multiple modeling experiments (clustering, LightGBM, histogram-based gradient boosting, Random Forest), and finally present findings in a PDF.
High-level components
a. Data Ingestion
• External Dataset (Google Drive link)
• Local storage (raw CSV/Parquet files)
b. Compute Environment
• Jupyter Notebook Server
• Python 3.x environment (pandas, numpy, scikit-learn, lightgbm, etc.)
c. Pipeline Stages (notebook modules)
1. Project_EDA.ipynb
– Exploratory Data Analysis → visualizations, summary stats
2. Preprocessing.ipynb
– Data cleaning, feature engineering → output: processed dataset
3. Modeling Branches (under “ML Models” folder)
• Clustering_LightGBM.ipynb → unsupervised clustering
• HistGBClassifier.ipynb → histogram-based gradient boosting classification
• RandomForest-Meet.ipynb → Random Forest predictions
d. Results/Outputs
• Trained model artifacts (pickles or in-notebook objects)
• BeyondTheTap_presentation.pdf (final slide deck)
Architectural patterns and design principles
• Linear pipeline with branching for parallel experiments
• Modular separation: each notebook is a discrete component (increases reproducibility)
• Clear data flow: raw → processed → model → report
• Environment as a shared “execution platform” component
• to “EDA Notebook”
• to “Preprocessing Notebook”
• Clustering_LightGBM
• HistGBClassifier
• RandomForest-Meet
5. Each model node outputs to a shared “Model Results & Metrics” node.
7. Enclose all notebooks within a “Jupyter Server” boundary to show common execution environment.