How saramansuri12/hinglish-sentiment works
This repository builds three-class sentiment classifiers from labeled text. The workflow converts CoNLL data to CSV, cleans the text, then trains and evaluates either TF-IDF-based classifiers or a fine-tuned XLM-R transformer. Classical scripts save model/vectorizer artifacts; the transformer script saves its model and tokenizer under models/xlm_roberta (that output directory is not in the supplied tree). The README is empty, so capabilities and links here are based on the tree and sampled source; no inference or serving workflow is evident.
Main entry points: train_word_char.py Word-char model · train_ml.py Logistic regression · train_svm.py Linear SVM · train_clean.csv Clean training CSV
Components 22
Dataset preparation
- Raw training data
data/raw/train_14k_split_conll.txt - Raw development data
data/raw/dev_3k_split_conll.txt - CoNLL conversion
convert_dataset.py - Training CSV
data/processed/train.csv - Development CSV
data/processed/dev.csv - Text cleaning
src/preprocess.py - Clean training CSV
data/processed/train_clean.csv - Clean development CSV
data/processed/dev_clean.csv
Classical models
- Logistic regression
src/train_ml.py - Linear SVM
src/train_svm.py - Word-char model
src/train_word_char.py - Dev evaluation
src/train_ml.py
Transformer model
- XLM-R fine-tuning
src/train_transformer.py
Saved artifacts
- Logistic model
models/logistic_regression.pkl - TF-IDF vectorizer
models/tfidf_vectorizer.pkl - SVM model
models/svm_model.pkl - SVM vectorizer
models/svm_tfidf_vectorizer.pkl - Word-char logistic
models/word_char_logistic.pkl - Word vectorizer
models/word_tfidf_vectorizer.pkl - Character vectorizer
models/char_tfidf_vectorizer.pkl
Other components
- Workflow operator
- Hugging Face model hub
Connections 34
- Workflow operator → CoNLL conversion: runs conversion
- Raw training data → CoNLL conversion: reads
- Raw development data → CoNLL conversion: reads
- CoNLL conversion → Training CSV: writes
- CoNLL conversion → Development CSV: writes
- Workflow operator → Text cleaning: runs cleaning
- Training CSV → Text cleaning: reads
- Development CSV → Text cleaning: reads
- Text cleaning → Clean training CSV: writes
- Text cleaning → Clean development CSV: writes
- Workflow operator → Logistic regression: runs training
- Workflow operator → Linear SVM: runs training
- Workflow operator → Word-char model: runs training
- Clean training CSV → Logistic regression: reads
- Clean development CSV → Logistic regression: reads
- Clean training CSV → Linear SVM: reads
- Clean development CSV → Linear SVM: reads
- Clean training CSV → Word-char model: reads
- Clean development CSV → Word-char model: reads
- Logistic regression → Dev evaluation: evaluates
- Linear SVM → Dev evaluation: evaluates
- Word-char model → Dev evaluation: evaluates
- Logistic regression → Logistic model: saves
- Logistic regression → TF-IDF vectorizer: saves
- Linear SVM → SVM model: saves
- Linear SVM → SVM vectorizer: saves
- Word-char model → Word-char logistic: saves
- Word-char model → Word vectorizer: saves
- Word-char model → Character vectorizer: saves
- Workflow operator → XLM-R fine-tuning: runs fine-tuning
- Clean training CSV → XLM-R fine-tuning: reads
- Clean development CSV → XLM-R fine-tuning: reads
- XLM-R fine-tuning → Hugging Face model hub: loads pretrained model
- XLM-R fine-tuning → Dev evaluation: evaluates