How acj08/thesis-project works
This repository combines a three-class lyric explicitness classifier with dataset-integrity tools and an iterative pseudo-labeling workflow. The main user path is to supply train/validation/test CSVs, train and select a model using validation results, evaluate on held-out test data, then classify lyrics and inspect LIME/SHAP explanations. The README describes the classifier’s model stages; excerpts show its multi-architecture implementation and parts of the pseudo-labeling and audit logic. Some workflow links rely on README documentation because the relevant implementation sections were not sampled.
Main entry points: data CSV splits · pseudo_labeling_best_teacher_fixed.py Pseudo-label training · lyric_classifier_v25_all_models.py Model variants · merge_labeled_songs.py Dataset merge
Components 24
Classification
- Prepare inputs
lyric_classifier_v25_all_models.py - Dataset loader
lyric_classifier_v25_all_models.py - RoBERTa encoder
lyric_classifier_v25_all_models.py - CNN–LSTM head
lyric_classifier_v25_all_models.py - Model variants
lyric_classifier_v25_all_models.py - Train and select
lyric_classifier_v25_all_models.py - Test evaluation
lyric_classifier_v25_all_models.py - Saved checkpoint
lyric_classifier_v25_all_models.py - Lyric inference
lyric_classifier_v25_all_models.py - LIME and SHAP
lyric_classifier_v25_all_models.py
Pseudo-labeling
- Pseudo-label training
pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py - Human review queue
pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py - Provisional data
pseudolabeling loop/data - Dataset merge
pseudolabeling loop/merge_labeled_songs.py - Merged exports
pseudolabeling loop/data
Dataset integrity
- Training deduplication
remove_training_duplicates_v3.py - Duplicate reports
training_duplicate_removals.csv - Split collision audit
inspect_split_identity_collisions.py - Collision report
split_identity_collisions.csv - Split overlap correction
remove_validation_test_overlap_fixed.py
Datasets
- CSV splits
data
Other components
- User
- Hugging Face
- Human reviewer
Connections 29
- User → CSV splits: provides
README.md - CSV splits → Prepare inputs: loads
README.md - Prepare inputs → Dataset loader: prepares
lyric_classifier_v25_all_models.py - Dataset loader → Train and select: supplies batches
README.md - Train and select → Model variants: trains
README.md - Model variants → RoBERTa encoder: encodes context
README.md - RoBERTa encoder → CNN–LSTM head: passes representations
README.md - CNN–LSTM head → Model variants: feeds logits
README.md - Model variants → Hugging Face: downloads pretrained model
README.md - Train and select → Saved checkpoint: saves best model
README.md - Saved checkpoint → Test evaluation: reloads for test
README.md - Test evaluation → Lyric inference: continues to examples
README.md - Lyric inference → LIME and SHAP: explains prediction
README.md - User → Pseudo-label training: runs iteration
pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py - Provisional data → Pseudo-label training: provides candidate data
- Pseudo-label training → Human review queue: appends flagged predictions
pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py - Human reviewer → Human review queue: reviews labels
pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py - Human review queue → Pseudo-label training: reads corrections
pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py - Provisional data → Dataset merge: reads provisional splits
pseudolabeling loop/merge_labeled_songs.py - Pseudo-label training → Dataset merge: produces training input
- Dataset merge → Merged exports: writes exports
pseudolabeling loop/merge_labeled_songs.py - User → Training deduplication: runs audit
remove_training_duplicates_v3.py - CSV splits → Training deduplication: reads splits
remove_training_duplicates_v3.py - Training deduplication → Duplicate reports: writes reports
remove_training_duplicates_v3.py - User → Split collision audit: runs audit
inspect_split_identity_collisions.py - CSV splits → Split collision audit: reads splits
inspect_split_identity_collisions.py - Split collision audit → Collision report: writes report
inspect_split_identity_collisions.py - User → Split overlap correction: runs correction
remove_validation_test_overlap_fixed.py - CSV splits → Split overlap correction: checks validation and test
remove_validation_test_overlap_fixed.py