How saiteja007-mv/cyberbully-detection-in-texts-images-and-audios works
Main Components
• Data Sources:
• Text Data: The project uses "twitter.csv" for cyberbullying classification.
• Image Data: Stored in the "images_dataset" folder (with "train" and "test" subdirectories that differentiate between "bully" and "non_bully" categories).
• Audio Data: Located in the "audios" directory, containing multiple audio samples.
• Preprocessing and Data Handling:
• Each modality (text, images, and audios) will have its preprocessing pipeline (e.g., tokenization for text using the tokenizer.json, image resizing/normalization, audio feature extraction or transcription).
• Model Training and Inference:
• Text Models: Utilize Bi-LSTM (evidenced by notebooks like Final_text_model.ipynb and models/cyberbullying_model_bilstm.h5).
• Image Models: Use CNN-based architectures (as seen from models/cyberbullying_model_CNN_image.h5 and Final_image_model.ipynb).
• Audio Analysis: While not as detailed in terms of specific models in the file tree, the README indicates that speech recognition and NLP techniques are used for audio analysis.
• Ensemble Approach: The README highlights that the project combines the outputs of the Bi-LSTM and CNN models to improve overall prediction accuracy.
• Evaluation and Visualization:
• Graphs and evaluation metrics (accuracy, precision, recall, confusion matrix) are captured and stored in the "Graphs" folder, with visualizations both for text and images.
• Documentation and Reporting:
• Documentation files (Cyberbullying_Documentation.docx/pdf, JETIR paper.pdf) provide further context on methodologies and results.
Relationships and Interactions
• Data Flow:
• For each modality, raw data is ingested (from CSV, image folders, and audio files).
• Preprocessing steps feed the cleaned and structured data into their respective pipelines.
• The training modules (Jupyter notebooks) perform model training and, subsequently, generate evaluation metrics.
• The ensemble mechanism takes outputs from the Bi-LSTM (text) and CNN (image) models to deliver an enriched detection output. Although audio processing might work independently, its results could also be integrated alongside text/image outcomes.
• Interaction Layers:
• There should be a clear distinction between the training phase and the inference/production pipeline.
Architectural Patterns and Design Principles
• Modular and Layered Design:
• Separate modules for handling each data modality.
• Distinct layers for Preprocessing, Model Training, and Evaluation.
• Clear Separation of Concerns:
• Each modality, though part of the ensemble, has its own dedicated processing flow. This helps in debugging, future enhancements, or integrating an API for real-time analysis as mentioned in the Future Work section.
• Integration of Ensemble Learning:
• Combination of Bi-LSTM (text-based) and CNN (image-based) outputs. Consider showing a decision layer or aggregator that combines these modalities.
Relevant Technologies and Tools
• Machine Learning & Deep Learning Frameworks: Likely using Keras/TensorFlow (based on model file types like .h5).
• Jupyter Notebooks: For code execution and experimentation.
• Audio Processing/NLP Libraries: For speech recognition, though specific tool names are not provided.
• Visualization and Evaluation: Graph generation for metrics (using libraries such as matplotlib or seaborn).
Detailing the System Design
• At the top level, start with three parallel data input streams:
• Text Input: “twitter.csv” → Text Preprocessor → Bi-LSTM Model Training & Inference.
• Image Input: “images_dataset” → Image Preprocessor → CNN Model Training & Inference.
• Audio Input: “audios” → Audio Feature Extraction/Preprocessor → Audio Analysis Pipeline (Speech Recognition/NLP).
• Next, connect these streams to an Ensemble Layer:
• How the outputs of the text and image models (and possibly audio results) combine to render a final decision.
• This module includes generating graphs, storing metrics, and summarizing performance (as seen in the "Graphs" directory).