How aditya-khade/final works
Project Type and Purpose
• This is a full-stack machine-learning web application for hate-speech detection.
• It ingests text datasets, trains and stores LSTM+CNN models, serves inference via a Flask API, and provides both interactive dashboards and a web UI for exploration.
File Structure
Top-level:
• Data/ – Raw and labeled CSV/XLSX datasets
• src/ – All Python code, entry point, and generated dashboards
– app.py: Flask server (API + UI)
– hate_speech_model*.py, comprehensive_hate_speech.py, religious_hate_speech.py: Inference modules
– train_models.py, hyperparameter_tuning.py: Model-training pipelines
– generate_visualizations.py & data_visualizations.py: Visualization‐generation scripts
– visualizations/: Prebuilt HTML dashboards
– vocab*.pkl, hate_speech_model_lstm_cnn.pth: Serialized artifacts
– requirements.txt: Dependencies
README
• Confirms Python+Flask stack, PyTorch models (LSTM+CNN), NLTK, Matplotlib/Seaborn dashboards
• Describes three main use cases: training, live analysis (/api/analyze), dashboard browsing
A. Data Layer
• Raw datasets (Data/*.csv, labeled_data.*)
• Vocabulary pickles (vocab.pkl, vocab_improved.pkl)
B. Training Pipeline
• Scripts: train_models.py, hyperparameter_tuning.py
• Core model code: hate_speech_model.py
• Output: serialized PyTorch model (.pth)
C. Visualization Pipeline
• Scripts: generate_visualizations.py, data_visualizations.py
• Input: raw datasets
• Output: HTML dashboards in src/visualizations/
D. Model Repository
• Stored artifacts: .pth, .pkl files
E. Web/API Layer
• Flask app (app.py)
• Routes:
– /api/analyze → comprehensive_hate_speech.py + model inference
– /visualizations → serve static dashboards
– / (UI) → text analysis form, dashboards navigator
F. Presentation Layer
• Client browser UI
• Interactive dashboards (HTML + JS)
Component Relationships and Data Flow
1. Raw Data → Training Pipeline → Trained Model → Model Repository
2. Raw Data → Visualization Pipeline → HTML Dashboards → Served by Flask
3. Client Text Input → Flask /api/analyze → Inference Modules → Model Repository → Prediction → Client UI
4. Client Browser → Flask /visualizations → Static Dashboard Files
Architectural Patterns and Principles
• Separation of concerns: Data, Training, Inference, Visualization, Presentation
• Pipeline pattern for both model training and dashboard generation
• MVC-style organization in Flask:
• Model: PyTorch artifacts + inference code
• View: HTML templates + dashboards
• Controller: app.py routes
• Modularity: each category of hate speech handled by dedicated module
Technologies and Frameworks
• Python 3.7+, Flask (web server)
• PyTorch (deep learning)
• NLTK (text preprocessing)
• Matplotlib/Seaborn, Plotly/Panel (dashboards)
• Pandas (data ingestion)
• HTML/CSS/JS (presentation)