# aditya-khade/final architecture

> Architecture diagram of the GitHub repository aditya-khade/final, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aditya-khade/final
- Repository: https://github.com/aditya-khade/final

## Overview

##### Project Type and Purpose
- This is a full-stack machine-learning web application for hate-speech detection.
- It ingests text datasets, trains and stores LSTM+CNN models, serves inference via a Flask API, and provides both interactive dashboards and a web UI for exploration.

##### File Structure
Top-level:
- Data/ – Raw and labeled CSV/XLSX datasets
- src/ – All Python code, entry point, and generated dashboards
– app.py: Flask server (API + UI)
– hate_speech_model*.py, comprehensive_hate_speech.py, religious_hate_speech.py: Inference modules
– train_models.py, hyperparameter_tuning.py: Model-training pipelines
– generate_visualizations.py & data_visualizations.py: Visualization‐generation scripts
– visualizations/: Prebuilt HTML dashboards
– vocab*.pkl, hate_speech_model_lstm_cnn.pth: Serialized artifacts
– requirements.txt: Dependencies

##### README
- Confirms Python+Flask stack, PyTorch models (LSTM+CNN), NLTK, Matplotlib/Seaborn dashboards
- Describes three main use cases: training, live analysis (/api/analyze), dashboard browsing

A. Data Layer
- Raw datasets (Data/*.csv, labeled_data.*)
- Vocabulary pickles (vocab.pkl, vocab_improved.pkl)
B. Training Pipeline
- Scripts: train_models.py, hyperparameter_tuning.py
- Core model code: hate_speech_model.py
- Output: serialized PyTorch model (.pth)
C. Visualization Pipeline
- Scripts: generate_visualizations.py, data_visualizations.py
- Input: raw datasets
- Output: HTML dashboards in src/visualizations/
D. Model Repository
- Stored artifacts: .pth, .pkl files
E. Web/API Layer
- Flask app (app.py)
- Routes:
– /api/analyze → comprehensive_hate_speech.py + model inference
– /visualizations → serve static dashboards
– / (UI) → text analysis form, dashboards navigator
F. Presentation Layer
- Client browser UI
- Interactive dashboards (HTML + JS)

##### Component Relationships and Data Flow
1. Raw Data → Training Pipeline → Trained Model → Model Repository
2. Raw Data → Visualization Pipeline → HTML Dashboards → Served by Flask
3. Client Text Input → Flask /api/analyze → Inference Modules → Model Repository → Prediction → Client UI
4. Client Browser → Flask /visualizations → Static Dashboard Files

##### Architectural Patterns and Principles
- Separation of concerns: Data, Training, Inference, Visualization, Presentation
- Pipeline pattern for both model training and dashboard generation
- MVC-style organization in Flask:
- Model: PyTorch artifacts + inference code
- View: HTML templates + dashboards
- Controller: app.py routes
- Modularity: each category of hate speech handled by dedicated module

##### Technologies and Frameworks
- Python 3.7+, Flask (web server)
- PyTorch (deep learning)
- NLTK (text preprocessing)
- Matplotlib/Seaborn, Plotly/Panel (dashboards)
- Pandas (data ingestion)
- HTML/CSS/JS (presentation)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion"
        direction TB
        raw1["Dataset_1.csv"]:::dataStore
        raw2["Dataset_2.csv"]:::dataStore
        raw3["Dataset_3.csv"]:::dataStore
        labcsv["labeled_data.csv"]:::dataStore
        labxlsx["labeled_data.xlsx"]:::dataStore
        labp["labeled_data.p"]:::dataStore
        vocab1["vocab.pkl"]:::dataStore
        vocab2["vocab_improved.pkl"]:::dataStore
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        train["train_models.py"]:::process
        tune["hyperparameter_tuning.py"]:::process
        coreModel["hate_speech_model.py"]:::process
    end

    %% Visualization Pipeline
    subgraph "Visualization Pipeline"
        direction TB
        genVis["generate_visualizations.py"]:::process
        dataVis["data_visualizations.py"]:::process
    end

    %% Model Store
    subgraph "Model Store"
        direction TB
        modelpth["hate_speech_model_lstm_cnn.pth"]:::dataStore
        storeV1["vocab.pkl"]:::dataStore
        storeV2["vocab_improved.pkl"]:::dataStore
    end

    %% API/Web Server
    subgraph "API/Web Server"
        direction TB
        flask["app.py"]:::external
        compHS["comprehensive_hate_speech.py"]:::process
        relHS["religious_hate_speech.py"]:::process
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        dashboard1["Dataset_1 dashboard.html"]:::dashboard
        dashboard2["Dataset_2 dashboard.html"]:::dashboard
        dashboard3["Dataset_3 dashboard.html"]:::dashboard
        dashboard4["labeled_data dashboard.html"]:::dashboard
        clientUI["Client Browser UI"]:::dashboard
    end

    %% Data Flow: Training
    raw1 -->|"train"| train
    raw2 -->|"train"| train
    raw3 -->|"train"| train
    labcsv -->|"train"| train
    labxlsx -->|"train"| train
    labp -->|"train"| train
    vocab1 -->|"train"| train
    vocab2 -->|"train"| train
    train -->|"serialize_model"| modelpth
    train -->|"serialize_vocab"| storeV1
    train -->|"serialize_vocab"| storeV2
    tune -->|"tune"| train
    coreModel -->|"used_by"| train

    %% Data Flow: Visualization
    raw1 -->|"visualize"| genVis
    raw2 -->|"visualize"| genVis
    raw3 -->|"visualize"| genVis
    labcsv -->|"visualize"| genVis
    labxlsx -->|"visualize"| genVis
    labp -->|"visualize"| genVis
    genVis -->|"generate"| dataVis
    dataVis -->|"output_dashboards"| dashboard1
    dataVis -->|"output_dashboards"| dashboard2
    dataVis -->|"output_dashboards"| dashboard3
    dataVis -->|"output_dashboards"| dashboard4

    %% Inference Flow
    clientUI -->|"HTTP POST /api/analyze"| flask
    flask -->|"load_model"| compHS
    flask -->|"load_model"| relHS
    compHS -->|"infer"| flask
    relHS -->|"infer"| flask
    flask -->|"respond"| clientUI

    %% Serve Dashboards
    clientUI -->|"GET /visualizations"| flask
    flask -->|"serve_static"| dashboard1
    flask -->|"serve_static"| dashboard2
    flask -->|"serve_static"| dashboard3
    flask -->|"serve_static"| dashboard4

    %% Serve UI
    clientUI -->|"GET /"| flask

    %% Click Events
    click raw1 "https://github.com/aditya-khade/final/blob/master/Data/Dataset_1.csv"
    click raw2 "https://github.com/aditya-khade/final/blob/master/Data/Dataset_2.csv"
    click raw3 "https://github.com/aditya-khade/final/blob/master/Data/Dataset_3.csv"
    click labcsv "https://github.com/aditya-khade/final/blob/master/Data/labeled_data.csv"
    click labxlsx "https://github.com/aditya-khade/final/blob/master/Data/labeled_data.xlsx"
    click labp "https://github.com/aditya-khade/final/blob/master/Data/labeled_data.p"
    click vocab1 "https://github.com/aditya-khade/final/blob/master/src/vocab.pkl"
    click vocab2 "https://github.com/aditya-khade/final/blob/master/src/vocab_improved.pkl"
    click train "https://github.com/aditya-khade/final/blob/master/src/train_models.py"
    click tune "https://github.com/aditya-khade/final/blob/master/src/hyperparameter_tuning.py"
    click coreModel "https://github.com/aditya-khade/final/blob/master/src/hate_speech_model.py"
    click modelpth "https://github.com/aditya-khade/final/blob/master/src/hate_speech_model_lstm_cnn.pth"
    click storeV1 "https://github.com/aditya-khade/final/blob/master/src/vocab.pkl"
    click storeV2 "https://github.com/aditya-khade/final/blob/master/src/vocab_improved.pkl"
    click genVis "https://github.com/aditya-khade/final/blob/master/src/generate_visualizations.py"
    click dataVis "https://github.com/aditya-khade/final/blob/master/src/data_visualizations.py"
    click dashboard1 "https://github.com/aditya-khade/final/blob/master/src/visualizations/Dataset_1/dashboard.html"
    click dashboard2 "https://github.com/aditya-khade/final/blob/master/src/visualizations/Dataset_2/dashboard.html"
    click dashboard3 "https://github.com/aditya-khade/final/blob/master/src/visualizations/Dataset_3/dashboard.html"
    click dashboard4 "https://github.com/aditya-khade/final/blob/master/src/visualizations/labeled_data/dashboard.html"
    click flask "https://github.com/aditya-khade/final/blob/master/src/app.py"
    click compHS "https://github.com/aditya-khade/final/blob/master/src/comprehensive_hate_speech.py"
    click relHS "https://github.com/aditya-khade/final/blob/master/src/religious_hate_speech.py"

    %% Styles
    classDef dataStore fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef process fill:#d4edda,stroke:#333,stroke-width:1px
    classDef dashboard fill:#ffe5b4,stroke:#333,stroke-width:1px
    classDef external fill:#e2e3e5,stroke:#333,stroke-width:1px
```
