# itsvenu/datascience-projects architecture

> Architecture diagram of the GitHub repository itsvenu/datascience-projects, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/itsvenu/datascience-projects
- Repository: https://github.com/itsvenu/datascience-projects

## Overview

##### Project type and purpose
- This repo is not a single deployed application but a collection of standalone data‐science/ML and data‐visualization “mini‐projects.” Each top‐level folder (AutoZOOM, Biotech_Salaries, COVID-19, etc.) encapsulates an independent analysis pipeline: ingest data, run scripts/notebooks, produce visuals, and link to a blog post.

##### File structure
- Top-level folders = individual projects.
- Within each project:
- data or data/ (raw inputs: CSV, Excel, Kaggle exports)
- scripts/ or directly R/Python script files & Jupyter notebooks (analysis code)
- results/ or output files (PDFs, PNGs, text summaries)
- Supporting files: README.md (global index), .gitignore, .DS_Store

##### Key insights from the README
- Describes each project’s purpose, blog‐post URL, and category (ML & Data Science vs DataViz).
- No centralized orchestration: each project is run independently and published separately.

a. Main components (for each mini‐project):
- External Data Sources (Kaggle, survey Excel, IMDB, Zoom metadata)
- Data Extraction & Cleaning Scripts (R scripts, Python .py files, notebooks)
- Analysis & Modeling (notebooks running statistical tests, ML algorithms)
- Visualization Outputs (PDFs, PNGs in results/)
- Blog Publishing Platform (the static site/Markdown that hosts the write‐ups)

b. Relationships & interactions
- That each mini‐project is isolated (no cross‐dependencies) but follows the same pipeline pattern.

c. Architectural patterns & design principles
- Modular isolation: each project as its own “service” or pipeline
- Reproducibility: raw data separated from code and outputs
- Convention over configuration: uniform folder layout across projects

d. Technologies & frameworks
- R (scripts .R, .RData outputs)
- Python (scripts .py, Jupyter .ipynb)
- Data storage: CSV, XLSX
- Outputs: PDF, PNG, text
- Version control: Git
- Blog: static site generator (e.g., Jekyll or Hugo inferred from .md blog links)

##### Elements
- External Data Sources
- Data folder (raw)
- Code folder (scripts & notebooks)
- Results folder (visuals & reports)
- Blog/Website
- Blue for Data
- Green for Code
- Orange for Outputs
- Purple for Publication
- Rectangles for Processes (scripts/notebooks)

##### Detailed capture of essential elements
- For each mini‐project, show the 4‐stage pipeline
- Special step (e.g., model training in BreastCancer_RandomForest, clustering in cb-challenge)
- Call out reusable patterns (e.g., Jupyter notebooks used for exploration, R scripts for batch jobs)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top-level aggregator
    README["README.md"]:::publication

    %% AutoZOOM Pipeline
    subgraph "AutoZOOM Pipeline"
        direction TB
        AZ_Data["External Data"]:::data
        AZ_Notebook["AutoZOOM_GetCoordinates.ipynb"]:::code
        AZ_Script["AutoZOOM_v0.03.py"]:::code
        AZ_Output["Results"]:::output
        AZ_Data -->|ingest| AZ_Notebook
        AZ_Notebook -->|process| AZ_Script
        AZ_Script -->|produce| AZ_Output
    end
    click AZ_Notebook "https://github.com/itsvenu/datascience-projects/blob/master/AutoZOOM/AutoZOOM_GetCoordinates.ipynb"
    click AZ_Script "https://github.com/itsvenu/datascience-projects/blob/master/AutoZOOM/AutoZOOM_v0.03.py"
    click AZ_Data "https://github.com/itsvenu/datascience-projects/tree/master/AutoZOOM/"
    
    %% Biotech Salaries Pipeline
    subgraph "Biotech Salaries Pipeline"
        direction TB
        BS_Data["Data Folder"]:::data
        BS_Clean1["01_making_money.R"]:::code
        BS_Clean2["01_making_money_v02.R"]:::code
        BS_Analysis["02_company_review.ipynb"]:::code
        BS_Output["Results Folder"]:::output
        BS_Data -->|clean| BS_Clean1
        BS_Clean1 -->|refine| BS_Clean2
        BS_Clean2 -->|analyze| BS_Analysis
        BS_Analysis -->|visualize| BS_Output
    end
    click BS_Data "https://github.com/itsvenu/datascience-projects/tree/master/Biotech_Salaries/data/"
    click BS_Clean1 "https://github.com/itsvenu/datascience-projects/blob/master/Biotech_Salaries/scripts/01_making_money.R"
    click BS_Clean2 "https://github.com/itsvenu/datascience-projects/blob/master/Biotech_Salaries/scripts/01_making_money_v02.R"
    click BS_Analysis "https://github.com/itsvenu/datascience-projects/blob/master/Biotech_Salaries/scripts/02_company_review.ipynb"
    click BS_Output "https://github.com/itsvenu/datascience-projects/tree/master/Biotech_Salaries/results/"

    %% Breast Cancer Random Forest Analysis
    subgraph "Breast Cancer Random Forest"
        direction TB
        BR_Data["External Data"]:::data
        BR_Notebook["BRCA_RandomForests.ipynb"]:::code
        BR_Presentation["RandomForests.pptx"]:::output
        BR_Data -->|train| BR_Notebook
        BR_Notebook -->|report| BR_Presentation
    end
    click BR_Notebook "https://github.com/itsvenu/datascience-projects/blob/master/BreastCancer_RandomForest/BRCA_RandomForests.ipynb"
    click BR_Presentation "https://github.com/itsvenu/datascience-projects/blob/master/BreastCancer_RandomForest/RandomForests.pptx"

    %% COVID-19 EDA
    subgraph "COVID-19 Exploratory Analysis"
        direction TB
        CV_Data["External Data"]:::data
        CV_Script["COVID-19-EDA.R"]:::code
        CV_Output["Results"]:::output
        CV_Data -->|process| CV_Script
        CV_Script -->|output| CV_Output
    end
    click CV_Script "https://github.com/itsvenu/datascience-projects/blob/master/COVID-19/COVID-19-EDA.R"

    %% Expenses Jan-Jun 2020
    subgraph "Expenses Jan–Jun 2020"
        direction TB
        EX_Data["External Data"]:::data
        EX_Script["expenses.R"]:::code
        EX_Output["Results"]:::output
        EX_Data -->|process| EX_Script
        EX_Script -->|produce| EX_Output
    end
    click EX_Script "https://github.com/itsvenu/datascience-projects/blob/master/Expenses_Jan2Jun2020/expenses.R"

    %% GOT Ratings Visualization
    subgraph "Game of Thrones Ratings"
        direction TB
        GOT_Data["External Data"]:::data
        GOT_Script["got_ratings.R"]:::code
        GOT_Output["GOT_Ratings.pdf"]:::output
        GOT_Data -->|analyze| GOT_Script
        GOT_Script -->|visualize| GOT_Output
    end
    click GOT_Script "https://github.com/itsvenu/datascience-projects/blob/master/GOT_Ratings/got_ratings.R"
    click GOT_Output "https://github.com/itsvenu/datascience-projects/blob/master/GOT_Ratings/GOT_Ratings.pdf"

    %% Google Location History Processing
    subgraph "Google Location History"
        direction TB
        GLH_Data["Raw Location Data"]:::data
        GLH_NB1["LatLon_to_city.ipynb"]:::code
        GLH_NB2["glh_22122018_R.ipynb"]:::code
        GLH_Output["Results"]:::output
        GLH_Data -->|convert| GLH_NB1
        GLH_NB1 -->|process| GLH_NB2
        GLH_NB2 -->|output| GLH_Output
    end
    click GLH_NB1 "https://github.com/itsvenu/datascience-projects/blob/master/GoogleLocationHistoty/LatLon_to_city.ipynb"
    click GLH_NB2 "https://github.com/itsvenu/datascience-projects/blob/master/GoogleLocationHistoty/glh_22122018_R.ipynb"

    %% India Temperature Trends
    subgraph "India Temperature Trends"
        direction TB
        IT_Data["temperatures.csv"]:::data
        IT_Script["india_temp.R"]:::code
        IT_Output["India_temperatarures.pdf"]:::output
        IT_Data -->|analyze| IT_Script
        IT_Script -->|report| IT_Output
    end
    click IT_Data "https://github.com/itsvenu/datascience-projects/blob/master/India_temperature/temperatures.csv"
    click IT_Script "https://github.com/itsvenu/datascience-projects/blob/master/India_temperature/india_temp.R"
    click IT_Output "https://github.com/itsvenu/datascience-projects/blob/master/India_temperature/India_temperatarures.pdf"

    %% Single-Cell Database Pipeline
    subgraph "Single-Cell Database"
        direction TB
        SC_Data["External Data"]:::data
        SC_Script["scDataBase_25052020.R"]:::code
        SC_Output["Results"]:::output
        SC_Data -->|process| SC_Script
        SC_Script -->|generate| SC_Output
    end
    click SC_Script "https://github.com/itsvenu/datascience-projects/blob/master/SingleCellDB/scDataBase_25052020.R"

    %% Women In Science Day Visualization
    subgraph "Women In Science Day"
        direction TB
        WD_Data["External Data"]:::data
        WD_Script["womenInScienceDay.R"]:::code
        WD_Output["Results"]:::output
        WD_Data -->|analyze| WD_Script
        WD_Script -->|visualize| WD_Output
    end
    click WD_Script "https://github.com/itsvenu/datascience-projects/blob/master/WomenInScienceDay/womenInScienceDay.R"

    %% Code-Basics Challenge
    subgraph "Code-Basics Challenge"
        direction TB
        CB_Data["numpyGeneratedKMneas_data.txt"]:::data
        CB_Scripts["*.R & *.ipynb"]:::code
        CB_Output["Results"]:::output
        CB_Data -->|use| CB_Scripts
        CB_Scripts -->|produce| CB_Output
    end
    click CB_Data "https://github.com/itsvenu/datascience-projects/blob/master/cb-challenge/numpyGeneratedKMneas_data.txt"
    click CB_Scripts "https://github.com/itsvenu/datascience-projects/tree/master/cb-challenge/"

    %% Global Blog Publication
    subgraph "Blog Publishing Platform"
        PUB["Website/Blog"]:::publication
    end

    %% Connections
    README --> AZ_Data
    README --> BS_Data
    README --> BR_Data
    README --> CV_Data
    README --> EX_Data
    README --> GOT_Data
    README --> GLH_Data
    README --> IT_Data
    README --> SC_Data
    README --> WD_Data
    README --> CB_Data
    AZ_Output --> PUB
    BS_Output --> PUB
    BR_Presentation --> PUB
    CV_Output --> PUB
    EX_Output --> PUB
    GOT_Output --> PUB
    GLH_Output --> PUB
    IT_Output --> PUB
    SC_Output --> PUB
    WD_Output --> PUB
    CB_Output --> PUB

    %% Styles
    classDef data fill:#B3DDF2,stroke:#2C7FB8,color:#000
    classDef code fill:#C6E3B5,stroke:#2A7F2A,color:#000
    classDef output fill:#FDD49E,stroke:#E34A33,color:#000
    classDef publication fill:#DADAEB,stroke:#756BB1,color:#000
```
