# aryan1524in/hate-speech-detection architecture

> Architecture diagram of the GitHub repository aryan1524in/hate-speech-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aryan1524in/hate-speech-detection
- Repository: https://github.com/aryan1524in/hate-speech-detection

## Overview

##### Project Type and Purpose
- This is a full-stack ML-powered web application for hate speech detection.
- Offline training pipeline (Jupyter notebook + CSV) produces a Decision Tree model with TF-IDF vectorizer, serialized as a pickle file.
- Online inference layer (Streamlit app) accepts user-typed text or YouTube comments (via YouTube Data API), runs preprocessing, profanity detection, then classification, and renders results.

##### Main Components
a. Offline Training Pipeline
- Data Sources:
– labeled_data.csv (training data)
– profanity_en.csv (optional feature)
- Preprocessing Module (Jupyter notebook – DT_edited.ipynb)
– Text cleaning, TF-IDF vectorizer fitting
- Model Training Module (Decision Tree)
- Model Artifact Store (modelfile.pkl)
b. Online Inference / Web App
- Streamlit Frontend (app.py)
- Inference Engine (final.py or internal functions)
– Load TF-IDF vectorizer + Decision Tree from modelfile.pkl
– Preprocess incoming text (same pipeline)
– Profanity filter lookup (profanity_en.csv)
– Run classification
- External Service Integration
– YouTube Data API (fetch comments by video ID)
- Result Renderer (Streamlit UI)
c. Auxiliary Files / Directories
- __pycache__ (runtime bytecode; not a functional component)

##### Relationships & Data Flow
- Offline pipeline writes modelfile.pkl → Inference Engine reads modelfile.pkl.
- User (browser) ↔ Streamlit Frontend
- Streamlit → Inference Engine → returns label/confidence
- For YouTube option: Streamlit Frontend → YouTube Data API → returns comments → Inference Engine
- Inference Engine uses profanity_en.csv as a lookup (side data file)

##### Architectural Patterns & Principles
- Layered Architecture:
1. Data Layer (CSV files, model file)
2. Processing Layer (preprocessing + inference logic)
3. Presentation Layer (Streamlit)
4. Integration Layer (YouTube API)
- Single codebase monolith, but logically separated into training vs inference.
- Reuse of preprocessing logic between training and inference ensures consistency.

##### Technologies & Frameworks
- Python 3.x
- scikit-learn (Decision Tree, TF-IDF)
- Streamlit (web UI)
- YouTube Data API (external service)
- CSV (data storage), pickle (model serialization)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Offline Training Pipeline"
        notebook["DT_edited.ipynb"]:::processing
        csv["labeled_data.csv"]:::data
        prof_csv["profanity_en.csv"]:::data
        model["modelfile.pkl"]:::data
        csv -->|"training data"| notebook
        prof_csv -->|"profanity list"| notebook
        notebook -->|"serialized model & vectorizer"| model
    end

    subgraph "Online Inference / Web App"
        user["User (Browser)"]:::external
        app["app.py\n(Streamlit UI)"]:::presentation
        engine["final.py\n(Inference Engine)"]:::processing
        model_artifact["modelfile.pkl"]:::data
        prof_lookup["profanity_en.csv"]:::data
        youtube["YouTube Data API"]:::external
        user -->|"text input\nor video ID"| app
        app -->|"inference request"| engine
        app -->|"video ID"| youtube
        youtube -->|"JSON comments"| engine
        engine -->|"loads model & vectorizer"| model_artifact
        engine -->|"uses profanity lookup"| prof_lookup
        engine -->|"predicted label & confidence"| app
    end

    subgraph "Auxiliary"
        aux["__pycache__/"]:::auxiliary
        readme["README.md"]:::auxiliary
    end

    click notebook "https://github.com/aryan1524in/hate-speech-detection/blob/main/DT_edited.ipynb"
    click csv "https://github.com/aryan1524in/hate-speech-detection/blob/main/labeled_data.csv"
    click prof_csv "https://github.com/aryan1524in/hate-speech-detection/blob/main/profanity_en.csv"
    click model "https://github.com/aryan1524in/hate-speech-detection/blob/main/modelfile.pkl"
    click app "https://github.com/aryan1524in/hate-speech-detection/blob/main/app.py"
    click engine "https://github.com/aryan1524in/hate-speech-detection/blob/main/final.py"
    click model_artifact "https://github.com/aryan1524in/hate-speech-detection/blob/main/modelfile.pkl"
    click prof_lookup "https://github.com/aryan1524in/hate-speech-detection/blob/main/profanity_en.csv"
    click aux "https://github.com/aryan1524in/hate-speech-detection/tree/main/__pycache__/"
    click readme "https://github.com/aryan1524in/hate-speech-detection/blob/main/README.md"

    classDef data fill:#ADD8E6,stroke:#000
    classDef processing fill:#90EE90,stroke:#000
    classDef presentation fill:#FFA500,stroke:#000
    classDef external fill:#DDA0DD,stroke:#000
    classDef auxiliary fill:#D3D3D3,stroke:#000
```
