# aatmanvaidya/sentiment-analysis-of-online-harassment-towards-women-wrestlers architecture

> Architecture diagram of the GitHub repository aatmanvaidya/sentiment-analysis-of-online-harassment-towards-women-wrestlers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aatmanvaidya/sentiment-analysis-of-online-harassment-towards-women-wrestlers
- Repository: https://github.com/aatmanvaidya/sentiment-analysis-of-online-harassment-towards-women-wrestlers

## Overview

##### Data Collection Layer
- YouTube Scraper (from youtube_scrape directory)
- Twitter Scraper (from twitter_scrape directory)
- Instagram Scraper (from insta_scrape directory)
- Data flow from social media platforms to the scrapers

##### Data Processing & Storage Layer
- A "Data Transformation" component showing:
- CSV to Feather conversion
- Video ID extraction
- Comment processing
- A "Data Storage" component showing the intermediate data formats

##### Analysis Layer
- BERTweet
- Dolly
- Google MuRiL
- Hinglish Model
- LLaMA v2
- Twitter RoBERTa
- A "Text Analysis" component showing:
- Regex Pattern Matching
- Slur Detection (using slur_list.txt)
- HurtLex Integration (using hurtlex_HI.tsv)

##### Output Layer
- A "Results & Analysis" component showing:
- Sentiment Analysis Output
- Harassment Detection Results
- Visualization components for the analysis results

- Data collection at the top
- Processing in the middle
- Analysis and results at the bottom

##### Additional Elements
- Input/output file formats (CSV, TSV, Feather, etc.)
- Integration points with external services
- Model dependencies and interactions

Key Considerations:

##### How different notebooks interact with each component

Remember to include version information for key components and any external dependencies that are crucial to the system's operation. This will help in understanding the system's requirements and maintenance needs.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles and Classes
    classDef scraper fill:#3498db,stroke:#2980b9,color:white
    classDef processing fill:#2ecc71,stroke:#27ae60,color:white
    classDef nlp fill:#f1c40f,stroke:#f39c12,color:black
    classDef analysis fill:#e74c3c,stroke:#c0392b,color:white
    classDef output fill:#9b59b6,stroke:#8e44ad,color:white
    classDef storage fill:#95a5a6,stroke:#7f8c8d,color:white

    %% Data Collection Layer
    subgraph DataCollection
        YT["YouTube Scraper"]:::scraper
        TW["Twitter Scraper"]:::scraper
        IG["Instagram Scraper"]:::scraper
    end

    %% Data Processing Layer
    subgraph DataProcessing
        CSV["CSV to Feather Conversion"]:::processing
        VID["Video ID Extraction"]:::processing
        CP["Comment Processing"]:::processing
        DS["Data Storage"]:::storage
    end

    %% NLP Models Layer
    subgraph NLPModels
        BT["BERTweet"]:::nlp
        DL["Dolly"]:::nlp
        GM["Google MuRiL"]:::nlp
        HM["Hinglish Model"]:::nlp
        LL["LLaMA v2"]:::nlp
        TR["Twitter RoBERTa"]:::nlp
    end

    %% Analysis Layer
    subgraph Analysis
        RPM["Regex Pattern Matching"]:::analysis
        SD["Slur Detection"]:::analysis
        HL["HurtLex Integration"]:::analysis
    end

    %% Output Layer
    subgraph Results
        SAO["Sentiment Analysis Output"]:::output
        HDR["Harassment Detection Results"]:::output
    end

    %% Connections
    YT --> CSV
    TW --> CSV
    IG --> CSV
    CSV --> DS
    VID --> DS
    DS --> CP
    CP --> NLPModels
    BT & DL & GM & HM & LL & TR --> Analysis
    Analysis --> Results

    %% Click Events
    click YT "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/tree/main/scraper/youtube_scrape"
    click TW "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/scraper/twitter_scrape/tweet_scrape_1.ipynb"
    click IG "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/scraper/insta_scrape/insta_scrape_1.ipynb"
    click CSV "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/scraper/csv_to_feather.ipynb"
    click VID "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/scraper/extract_video_id.ipynb"
    click CP "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/scraper/comments.py"
    click BT "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/nlp/bertweet.ipynb"
    click DL "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/nlp/dolly-nlp.ipynb"
    click GM "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/nlp/google-muril.ipynb"
    click HM "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/nlp/hinglish_model.ipynb"
    click LL "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/nlp/llama_v2.ipynb"
    click TR "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/nlp/twitter-roberta.ipynb"
    click SD "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/regex/slur_list.txt"
    click HL "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/regex/hurtlex_HI.tsv"
    click SAO "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/youtube_comments_analysis.ipynb"
    click HDR "https://github.com/aatmanvaidya/Sentiment-Analysis-of-Online-Harassment-Towards-Women-Wrestlers/blob/main/youtube_comments_analysis.ipynb"

    %% Legend
    subgraph Legend
        L1["Data Collection"]:::scraper
        L2["Data Processing"]:::processing
        L3["NLP Models"]:::nlp
        L4["Analysis"]:::analysis
        L5["Output"]:::output
        L6["Storage"]:::storage
    end
```
