How adithya-257/mental-health-analysis works
Purpose and Entry Points
This is an offline NLP/ML pipeline plus Streamlit dashboard for classifying Reddit posts into depression, anxiety, bipolar, PTSD, or normal. The operational entry point is src/app.py; pipeline stages are individually executable scripts, with the intended sequence documented in README.md.
Dataset Preparation and Text Boundaries
Raw train/validation/test CSVs are expected under an untracked data/ directory. src/build_clean_dataset.py creates reusable cleaned splits, using src/preprocess.py. Preprocessing intentionally branches:
• Transformer-safe cleaning preserves sentence structure for RoBERTa.
• Traditional normalization supports lexical analytics, VADER, and vectorizers.
This separation prevents destructive token cleanup from affecting transformer inputs.
Data Access and Shared Configuration
src/config.py centralizes model/training and path-level assumptions; src/data_loader.py loads dataset splits for downstream consumers. These are the main coordination boundary between file-based datasets and model/analysis stages. Paths referenced in the repository include src/config.py, src/data_loader.py, and src/build_clean_dataset.py.
Modeling: Baseline and Transformer
The primary classifier is a fine-tuned roberta-base five-class model implemented by src/train.py, with a published checkpoint hosted on Hugging Face (Adithya-257/mental-health-roberta). Training produces run and evaluation metadata in logs/runs.json and logs/eval_results.json. A separate TF-IDF plus logistic-regression comparison path is implemented in src/baseline.py; vector construction and serialized feature artifacts are handled by src/feature_extraction.py.
Evaluation and Explainability
src/evaluate.py evaluates the trained classifier on held-out data and emits reports, confusion/ROC visualizations, and row-level error analysis. Persisted outputs include logs/classification_report.txt, logs/eval_results.json, and assets/error_analysis.csv. src/lime_explainer.py wraps model inference for LIME, exporting human-readable HTML and visual explanations to assets/lime_html/ and assets/lime_examples/; existing explanation files demonstrate that output contract.
Parallel Sentiment and Corpus Analytics
VADER is an independent sentiment-analysis path, rather than an input feature required by RoBERTa, implemented in src/vader_module.py. Corpus-level lexical artifacts derive from traditional preprocessing: src/wordcloud_gen.py creates class word clouds in assets/wordclouds/, while src/frequent_words.py creates per-class term-frequency charts in assets/frequent_words/. Dashboard and evaluation charts are stored under assets/charts/.
Runtime, UI, and Dependencies
The system runs in Python with dependencies declared in requirements.txt; material libraries include Hugging Face transformer tooling, VADER, LIME, scikit-learn-style vectorization/baselines, and Streamlit. src/app.py is the serving boundary: it accepts a user-supplied post, invokes RoBERTa prediction and VADER sentiment in parallel presentation, and surfaces class probabilities and LIME explanation. It depends on local/generated artifacts and/or the Hugging Face model checkpoint; no backend service or database is represented in the tree.
Main entry points: data_loader.py Dataset loader · train.py RoBERTa training · build_clean_dataset.py Build clean dataset · app.py Streamlit dashboard
Components 21
Data preparation
File-based dataset and two text-processing paths.
- Raw CSV splits · Untracked train, validation, and test inputs.
- Build clean dataset · Creates reusable cleaned splits.
src/build_clean_dataset.py - Text preprocessing · Separates transformer-safe cleaning from traditional normalization.
src/preprocess.py - Cleaned splits · Shared inputs for model and analysis stages.
- Shared configuration · Centralizes paths and model/training assumptions.
src/config.py - Dataset loader · Loads splits for downstream consumers.
src/data_loader.py
Modeling and evaluation
Classifier training, comparison, evaluation, and explanations.
- RoBERTa training · Fine-tunes the five-class roberta-base classifier.
src/train.py - RoBERTa checkpoint · Published five-class model used for inference.
- Run metadata · Training and evaluation run records.
logs/runs.json - Feature extraction · Builds and serializes TF-IDF feature artifacts.
src/feature_extraction.py - TF-IDF baseline · Logistic-regression comparison path.
src/baseline.py - Model evaluation · Produces reports, metrics, plots, and row-level errors.
src/evaluate.py - Evaluation outputs · Report, metrics, charts, and error analysis.
logs/eval_results.json - LIME explainer · Wraps classifier inference for human-readable explanations.
src/lime_explainer.py - LIME outputs · HTML and visual explanation exports.
Analytics outputs
Independent sentiment and corpus-analysis pipeline.
- VADER sentiment · Independent sentiment-analysis path.
src/vader_module.py - Corpus lexical analytics · Generates class word clouds and frequent-term charts.
src/wordcloud_gen.py - Lexical artifacts · Per-class word clouds and term-frequency charts.
Dashboard runtime
Interactive serving boundary.
- Streamlit dashboard · Accepts posts and presents predictions, sentiment, probabilities, and explanations.
src/app.py - Python dependencies · Declares Streamlit, Transformers, VADER, LIME, and ML libraries.
requirements.txt
Other components
- Pipeline sequence · Documents the intended executable-stage order.
README.md
Connections 25
- Raw CSV splits → Build clean dataset: input
- Build clean dataset → Text preprocessing: cleans with
- Text preprocessing → Cleaned splits: produces
- Shared configuration → Build clean dataset: paths
- Shared configuration → Dataset loader: paths
- Cleaned splits → Dataset loader: loaded by
- Dataset loader → RoBERTa training: training data
- Shared configuration → RoBERTa training: training settings
- RoBERTa training → RoBERTa checkpoint: produces/publishes
- RoBERTa training → Run metadata: writes
- Dataset loader → Feature extraction: traditional text
- Feature extraction → TF-IDF baseline: TF-IDF features
- Dataset loader → Model evaluation: held-out data
- RoBERTa checkpoint → Model evaluation: evaluated model
- Model evaluation → Evaluation outputs: writes
- RoBERTa checkpoint → LIME explainer: inference wrapper
- LIME explainer → LIME outputs: exports
- Dataset loader → VADER sentiment: traditional text
- Dataset loader → Corpus lexical analytics: traditional text
- Corpus lexical analytics → Lexical artifacts: exports
- RoBERTa checkpoint → Streamlit dashboard: class prediction
- VADER sentiment → Streamlit dashboard: sentiment result
- LIME explainer → Streamlit dashboard: explanation
- Python dependencies → Streamlit dashboard: runtime dependencies
- Pipeline sequence → Build clean dataset: documents sequence