How vrishin-reddy/emotion_detection works
Project Type and Purpose
• This is a machine learning–based emotion detection system. It processes text through traditional ML algorithms (TF-IDF with models like Logistic Regression, SVM, etc.) as well as advanced deep learning (fine-tuned BERT) and then provides real-time predictions via a web interface, built using a Streamlit application.
• The project is aimed at analyzing input text, classifying it into emotion categories, and displaying interactive results and insights.
Analyzing the File Structure
• Notebooks (8_Mod.ipynb and BERT_Mod.ipynb): These files denote the data preprocessing, model training, hyperparameter tuning, and evaluation stages.
• Application Files (app.py and app_Streamlit.py): One of these (app_Streamlit.py) is explicitly noted as the Streamlit app for user interaction, while app.py might be another entry point.
• Model Saving and Evaluation (model_saves folder): This contains saved models, configuration files, metrics, and evaluation results (e.g., CSV files, JSON reports). This directory is key for showcasing how trained models and performance analytics are managed.
• Static and Templates (static, templates folders): These provide front-end assets (CSS for styling) and HTML views for web pages (error pages and index) which support the web interface.
• Support Files (requirements.txt, .devcontainer for development container): They indicate the dependencies and the development environment configuration.
Main Components
a. Data Input and Preprocessing:
• Input Text Data: Users send text input.
• Preprocessing Pipeline: TF-IDF vectorization, tokenization (using resources like tokenizer.json, tokenizer_config.json, and vocab.txt). This is described within the notebooks.
b. Machine Learning Module:
• Traditional ML Models: Models such as Logistic Regression, SVM, KNN, Decision Tree, Naïve Bayes, and Lasso Regression (as seen in the model_saves folder with their saved states and evaluation results).
• BERT Module: A fine-tuned BERT model (separated in its own notebook) that likely has additional handling for hyperparameter tuning and more accurate predictions.
• Model Evaluation and Cross-validation: Components that generate CSV/JSON reports (e.g., cross_validation_results, evaluation_metrics.json) to validate model performance.
c. Model Management:
• Model Storage: The model_saves directory acts as a repository that organizes models, metrics, and configuration files (config.json, special_tokens_map.json, tfidf_vect.pk, etc.).
• Evaluation Results: Stores both training and model evaluation metrics, guiding improvements and debugging.
d. Web Application (Frontend):
• Streamlit App: The app_Streamlit.py is responsible for interacting with the end-user, fetching predictions, and displaying model insights and visualizations.
• Templates and Static Assets: Serve as the HTML/CSS layer that supports the web interface (templates for error pages and main index).
e. Development/Deployment Environment:
• .devcontainer folder: Defines a containerized development environment to ensure consistency across setups.
Architectural Patterns and Design Principles
• Separation of Concerns: Split between data processing/model training (back-end ML pipelines) and the user interface (front-end via Streamlit). This decoupling allows independent updates.
• Modular Design: Each component (e.g., preprocess/notebooks, ML models, model evaluation, web server) is self-contained.