# nikhilsaraogi/hate-tweet-sentiment-analysis architecture

> Architecture diagram of the GitHub repository nikhilsaraogi/hate-tweet-sentiment-analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nikhilsaraogi/hate-tweet-sentiment-analysis
- Repository: https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis

## Overview

##### Project Type and Purpose
- This is a data‐science pipeline centered on sentiment analysis of tweets, built primarily with Jupyter notebooks and flat‐file datasets.
- Purpose: Collect raw tweets, preprocess and clean them, perform exploratory data analysis (EDA), train and validate classification models, then summarize findings.

##### File Structure
Top-level artifacts:
- EDA.ipynb   – Exploratory Data Analysis notebook
- Modelling.ipynb – Model training and evaluation notebook
- data_collection.ipynb – Tweet extraction/preprocessing notebook
- data/       – Raw and processed CSVs and pickled train/val splits
- Review.pptx – Final presentation of results
- README.md, LICENSE

##### Extract Insights from README
- Confirms pipeline stages: data ingestion (“extracting vast number of tweets”), binary sentiment classification, analysis of experiments, discussion of limitations.

##### Main System Components
a. Data Sources
– Twitter API (external) or flat CSVs if tweets were already collected
b. Ingestion & Preprocessing
– data_collection.ipynb: cleaning, tokenization, labeling
– Generates .csv or pickled train/validation sets
c. Data Storage
– data/ folder holding CSVs and pickles
d. Exploratory Data Analysis
– EDA.ipynb: statistical summaries, visualizations
e. Feature Engineering & Modeling
– Modelling.ipynb: vectorization, model training (e.g., scikit-learn, possibly deep learning libs)
f. Results & Reporting
– images/: plots saved from notebooks
– Review.pptx: slide deck summarizing results

##### Relationships & Data Flows
– External Twitter API → data_collection.ipynb → data/*.csv
– data/*.csv → data_collection.ipynb → train.pickle/val.pickle
– train/val pickles → Modelling.ipynb
– Raw CSVs → EDA.ipynb
– Notebooks output figures → images/
– Final insights → Review.pptx

##### Architectural Patterns & Principles
– Linear ETL pipeline: Extract → Transform → Load → Analyze → Model → Report
– Modular separation by notebook/functionality
– Flat-file storage for reproducibility
– Use of Python data-science stack (Pandas, scikit-learn, Matplotlib/Seaborn)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Service
    TwitterAPI["Twitter API"]:::external

    %% Data Ingestion & Preprocessing
    DataCollection["Data Collection\n(data_collection.ipynb)"]:::ingestion

    %% Data Storage with Stored Datasets
    subgraph "Data Storage" 
        DS[(Data Storage)]:::storage
        subgraph "Stored Datasets & Splits"
            balanced["balanced_data_combined.csv"]:::storage
            baseline["baseline_df.csv"]:::storage
            ras["df_ras_sex_hate.csv"]:::storage
            hate["hate_add.csv"]:::storage
            labeled["labeled_data.csv"]:::storage
            train["train.pickle"]:::storage
            val["val.pickle"]:::storage
        end
    end

    %% Exploratory Data Analysis
    EDA["EDA\n(EDA.ipynb)"]:::analysis

    %% Feature Engineering & Modeling
    Modelling["Modeling\n(Modelling.ipynb)"]:::modeling

    %% Visualization Artifacts
    Images["Visualizations\n(images/)"]:::analysis

    %% Reporting & Presentation
    Reporting["Reporting\n(Review.pptx)"]:::docs

    %% Documentation
    README["README.md"]:::docs
    LICENSE["LICENSE"]:::docs

    %% Data Flow
    TwitterAPI -->|"raw JSON"| DataCollection
    DataCollection -->|"cleaned CSV"| DS
    DataCollection -->|"train/val pickle"| DS
    DS -->|"CSV"| EDA
    DS -->|"pickle"| Modelling
    EDA -->|"plots"| Images
    Modelling -->|"model metrics & plots"| Images
    Images -->|"figures"| Reporting
    Reporting --> README
    Reporting --> LICENSE

    %% Click Events
    click DataCollection "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data_collection.ipynb"
    click EDA "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/EDA.ipynb"
    click Modelling "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/Modelling.ipynb"
    click DS "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/tree/main/data/"
    click balanced "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/balanced_data_combined.csv"
    click baseline "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/baseline_df.csv"
    click ras "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/df_ras_sex_hate.csv"
    click hate "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/hate_add.csv"
    click labeled "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/labeled_data.csv"
    click train "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/train.pickle"
    click val "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/data/val.pickle"
    click Images "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/tree/main/images/"
    click Reporting "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/Review.pptx"
    click README "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/blob/main/README.md"
    click LICENSE "https://github.com/nikhilsaraogi/hate-tweet-sentiment-analysis/tree/main/LICENSE"

    %% Styles
    classDef ingestion fill:#D0E8FF,stroke:#000;
    classDef analysis fill:#D0FFD6,stroke:#000;
    classDef modeling fill:#FFE0B3,stroke:#000;
    classDef storage fill:#E0E0E0,stroke:#000;
    classDef external fill:#F0F0F0,stroke:#000;
    classDef docs fill:#F5F5F5,stroke:#000;
```
