# anvayb/beyondthetap architecture

> Architecture diagram of the GitHub repository anvayb/beyondthetap, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anvayb/beyondthetap
- Repository: https://github.com/anvayb/beyondthetap

## Overview

##### Project type and purpose
- This is a data-science / machine-learning analysis project (not a client–server or microservice app).
- Its goal is to ingest water-quality data, perform exploratory data analysis (EDA), preprocess and engineer features, run multiple modeling experiments (clustering, LightGBM, histogram-based gradient boosting, Random Forest), and finally present findings in a PDF.

##### High-level components
a. Data Ingestion
- External Dataset (Google Drive link)
- Local storage (raw CSV/Parquet files)
b. Compute Environment
- Jupyter Notebook Server
- Python 3.x environment (pandas, numpy, scikit-learn, lightgbm, etc.)
c. Pipeline Stages (notebook modules)
1. Project_EDA.ipynb
– Exploratory Data Analysis → visualizations, summary stats
2. Preprocessing.ipynb
– Data cleaning, feature engineering → output: processed dataset
3. Modeling Branches (under “ML Models” folder)
- Clustering_LightGBM.ipynb → unsupervised clustering
- HistGBClassifier.ipynb → histogram-based gradient boosting classification
- RandomForest-Meet.ipynb → Random Forest predictions
d. Results/Outputs
- Trained model artifacts (pickles or in-notebook objects)
- BeyondTheTap_presentation.pdf (final slide deck)

##### Architectural patterns and design principles
- Linear pipeline with branching for parallel experiments
- Modular separation: each notebook is a discrete component (increases reproducibility)
- Clear data flow: raw → processed → model → report
- Environment as a shared “execution platform” component

- to “EDA Notebook”
- to “Preprocessing Notebook”
- Clustering_LightGBM
- HistGBClassifier
- RandomForest-Meet
5. Each model node outputs to a shared “Model Results & Metrics” node.
7. Enclose all notebooks within a “Jupyter Server” boundary to show common execution environment.

## Diagram (Mermaid)

```mermaid
flowchart TB
    External["External Dataset (Google Drive)"]:::external
    External -->|"downloads to"| Raw[(Raw Water Quality Data)]:::datastore

    subgraph "Jupyter Notebook Server"
        direction TB
        Env["Jupyter Notebook Server (Python 3.x + libraries)"]:::env
        Env --> EDA["EDA Module"]:::process
        Env --> Preprocess["Preprocessing Module"]:::process

        EDA -->|"explores raw data"| Summary["Summary Outputs"]:::output
        Preprocess -->|"cleans & engineers features"| Processed[(Processed Data Storage)]:::datastore

        Processed --> Cluster["Clustering Model"]:::model
        Processed --> Hist["HistGBClassifier"]:::model
        Processed --> RF["Random Forest Model"]:::model

        Cluster --> Results["Model Results & Metrics"]:::output
        Hist --> Results
        RF --> Results
    end

    Results -->|"compiles findings into"| SlideDeck["BeyondTheTap_presentation.pdf"]:::output

    click External "https://github.com/anvayb/beyondthetap/blob/main/README.md"
    click Env "https://github.com/anvayb/beyondthetap/blob/main/requirements.txt"
    click EDA "https://github.com/anvayb/beyondthetap/blob/main/ML Models/Project_EDA.ipynb"
    click Preprocess "https://github.com/anvayb/beyondthetap/blob/main/ML Models/Preprocessing.ipynb"
    click Cluster "https://github.com/anvayb/beyondthetap/blob/main/ML Models/Clustering_LightGBM.ipynb"
    click Hist "https://github.com/anvayb/beyondthetap/blob/main/ML Models/HistGBClassifier.ipynb"
    click RF "https://github.com/anvayb/beyondthetap/blob/main/ML Models/RandomForest-Meet.ipynb"
    click SlideDeck "https://github.com/anvayb/beyondthetap/blob/main/BeyondTheTap_presentation.pdf"

    classDef datastore fill:#ADD8E6,stroke:#000
    classDef process fill:#90EE90,stroke:#000
    classDef model fill:#FFA500,stroke:#000
    classDef output fill:#DDA0DD,stroke:#000
    classDef env fill:#f0f0f0,stroke:#000
    classDef external fill:#E0FFFF,stroke:#000
```
