# acj08/thesis-project architecture

> Architecture diagram of the GitHub repository acj08/thesis-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/acj08/thesis-project
- Repository: https://github.com/acj08/thesis-project

## Overview

This repository combines a three-class lyric explicitness classifier with dataset-integrity tools and an iterative pseudo-labeling workflow. The main user path is to supply train/validation/test CSVs, train and select a model using validation results, evaluate on held-out test data, then classify lyrics and inspect LIME/SHAP explanations. The README describes the classifier’s model stages; excerpts show its multi-architecture implementation and parts of the pseudo-labeling and audit logic. Some workflow links rely on README documentation because the relevant implementation sections were not sampled.

Main entry points: [`data`](https://github.com/acj08/thesis-project/tree/main/data) (CSV splits), [`pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`](https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/pseudo_labeling_best_teacher_fixed.py) (Pseudo-label training), [`lyric_classifier_v25_all_models.py`](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py) (Model variants), [`pseudolabeling loop/merge_labeled_songs.py`](https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/merge_labeled_songs.py) (Dataset merge)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_classification["Classification"]
  node_prepare["Prepare inputs"]
  node_dataset_loader["Dataset loader"]
  node_roberta["RoBERTa encoder"]
  node_cnn_lstm["CNN–LSTM head"]
  node_classifier["Model variants"]
  node_train_select["Train and select"]
  node_evaluate["Test evaluation"]
  node_saved_model["Saved checkpoint"]
  node_inference["Lyric inference"]
  node_explain["LIME and SHAP"]
end

subgraph group_pseudo["Pseudo-labeling"]
  node_pseudo_train["Pseudo-label training"]
  node_review_queue["Human review queue"]
  node_provisional["Provisional data"]
  node_merge["Dataset merge"]
  node_final_dataset["Merged exports"]
end

subgraph group_integrity["Dataset integrity"]
  node_dedupe["Training deduplication"]
  node_duplicate_reports["Duplicate reports"]
  node_collision_audit["Split collision audit"]
  node_collision_report["Collision report"]
  node_overlap_fix["Split overlap correction"]
end

subgraph group_data["Datasets"]
  node_csv_splits["CSV splits"]
end

node_user(("User"))
node_hf["Hugging Face"]
node_reviewer(("Human reviewer"))

node_user -->|"provides"| node_csv_splits
node_csv_splits -->|"loads"| node_prepare
node_prepare -->|"prepares"| node_dataset_loader
node_dataset_loader -->|"supplies batches"| node_train_select
node_train_select -->|"trains"| node_classifier
node_classifier -->|"encodes context"| node_roberta
node_roberta -->|"passes representations"| node_cnn_lstm
node_cnn_lstm -->|"feeds logits"| node_classifier
node_classifier -->|"downloads pretrained model"| node_hf
node_train_select -->|"saves best model"| node_saved_model
node_saved_model -->|"reloads for test"| node_evaluate
node_evaluate -->|"continues to examples"| node_inference
node_inference -->|"explains prediction"| node_explain
node_user -->|"runs iteration"| node_pseudo_train
node_provisional -.->|"provides candidate data"| node_pseudo_train
node_pseudo_train -->|"appends flagged predictions"| node_review_queue
node_reviewer -->|"reviews labels"| node_review_queue
node_review_queue -->|"reads corrections"| node_pseudo_train
node_provisional -->|"reads provisional splits"| node_merge
node_pseudo_train -.->|"produces training input"| node_merge
node_merge -->|"writes exports"| node_final_dataset
node_user -->|"runs audit"| node_dedupe
node_csv_splits -->|"reads splits"| node_dedupe
node_dedupe -->|"writes reports"| node_duplicate_reports
node_user -->|"runs audit"| node_collision_audit
node_csv_splits -->|"reads splits"| node_collision_audit
node_collision_audit -->|"writes report"| node_collision_report
node_user -->|"runs correction"| node_overlap_fix
node_csv_splits -->|"checks validation and test"| node_overlap_fix

click node_csv_splits "https://github.com/acj08/thesis-project/tree/main/data"
click node_prepare "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_dataset_loader "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_roberta "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_cnn_lstm "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_classifier "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_train_select "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_evaluate "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_saved_model "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_inference "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_explain "https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py"
click node_pseudo_train "https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/pseudo_labeling_best_teacher_fixed.py"
click node_review_queue "https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/pseudo_labeling_best_teacher_fixed.py"
click node_provisional "https://github.com/acj08/thesis-project/tree/main/pseudolabeling%20loop/data"
click node_merge "https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/merge_labeled_songs.py"
click node_final_dataset "https://github.com/acj08/thesis-project/tree/main/pseudolabeling%20loop/data"
click node_dedupe "https://github.com/acj08/thesis-project/blob/main/remove_training_duplicates_v3.py"
click node_duplicate_reports "https://github.com/acj08/thesis-project/blob/main/training_duplicate_removals.csv"
click node_collision_audit "https://github.com/acj08/thesis-project/blob/main/inspect_split_identity_collisions.py"
click node_collision_report "https://github.com/acj08/thesis-project/blob/main/split_identity_collisions.csv"
click node_overlap_fix "https://github.com/acj08/thesis-project/blob/main/remove_validation_test_overlap_fixed.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepare,node_dataset_loader,node_roberta,node_cnn_lstm,node_classifier,node_train_select,node_evaluate,node_saved_model,node_inference,node_explain,node_user,node_reviewer toneBlue
class node_pseudo_train,node_review_queue,node_provisional,node_merge,node_final_dataset toneAmber
class node_dedupe,node_duplicate_reports,node_collision_audit,node_collision_report,node_overlap_fix toneMint
class node_csv_splits toneRose
class node_hf toneIndigo
```

## Components

### Classification

- [Prepare inputs](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [Dataset loader](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [RoBERTa encoder](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [CNN–LSTM head](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [Model variants](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [Train and select](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [Test evaluation](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [Saved checkpoint](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [Lyric inference](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`
- [LIME and SHAP](https://github.com/acj08/thesis-project/blob/main/lyric_classifier_v25_all_models.py): `lyric_classifier_v25_all_models.py`

### Pseudo-labeling

- [Pseudo-label training](https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/pseudo_labeling_best_teacher_fixed.py): `pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`
- [Human review queue](https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/pseudo_labeling_best_teacher_fixed.py): `pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`
- [Provisional data](https://github.com/acj08/thesis-project/tree/main/pseudolabeling%20loop/data): `pseudolabeling loop/data`
- [Dataset merge](https://github.com/acj08/thesis-project/blob/main/pseudolabeling%20loop/merge_labeled_songs.py): `pseudolabeling loop/merge_labeled_songs.py`
- [Merged exports](https://github.com/acj08/thesis-project/tree/main/pseudolabeling%20loop/data): `pseudolabeling loop/data`

### Dataset integrity

- [Training deduplication](https://github.com/acj08/thesis-project/blob/main/remove_training_duplicates_v3.py): `remove_training_duplicates_v3.py`
- [Duplicate reports](https://github.com/acj08/thesis-project/blob/main/training_duplicate_removals.csv): `training_duplicate_removals.csv`
- [Split collision audit](https://github.com/acj08/thesis-project/blob/main/inspect_split_identity_collisions.py): `inspect_split_identity_collisions.py`
- [Collision report](https://github.com/acj08/thesis-project/blob/main/split_identity_collisions.csv): `split_identity_collisions.csv`
- [Split overlap correction](https://github.com/acj08/thesis-project/blob/main/remove_validation_test_overlap_fixed.py): `remove_validation_test_overlap_fixed.py`

### Datasets

- [CSV splits](https://github.com/acj08/thesis-project/tree/main/data): `data`

### Other components

- **User**
- **Hugging Face**
- **Human reviewer**

## Connections

- User → CSV splits: provides (evidence: `README.md`)
- CSV splits → Prepare inputs: loads (evidence: `README.md`)
- Prepare inputs → Dataset loader: prepares (evidence: `lyric_classifier_v25_all_models.py`)
- Dataset loader → Train and select: supplies batches (evidence: `README.md`)
- Train and select → Model variants: trains (evidence: `README.md`)
- Model variants → RoBERTa encoder: encodes context (evidence: `README.md`)
- RoBERTa encoder → CNN–LSTM head: passes representations (evidence: `README.md`)
- CNN–LSTM head → Model variants: feeds logits (evidence: `README.md`)
- Model variants → Hugging Face: downloads pretrained model (evidence: `README.md`)
- Train and select → Saved checkpoint: saves best model (evidence: `README.md`)
- Saved checkpoint → Test evaluation: reloads for test (evidence: `README.md`)
- Test evaluation → Lyric inference: continues to examples (evidence: `README.md`)
- Lyric inference → LIME and SHAP: explains prediction (evidence: `README.md`)
- User → Pseudo-label training: runs iteration (evidence: `pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`)
- Provisional data → Pseudo-label training: provides candidate data
- Pseudo-label training → Human review queue: appends flagged predictions (evidence: `pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`)
- Human reviewer → Human review queue: reviews labels (evidence: `pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`)
- Human review queue → Pseudo-label training: reads corrections (evidence: `pseudolabeling loop/pseudo_labeling_best_teacher_fixed.py`)
- Provisional data → Dataset merge: reads provisional splits (evidence: `pseudolabeling loop/merge_labeled_songs.py`)
- Pseudo-label training → Dataset merge: produces training input
- Dataset merge → Merged exports: writes exports (evidence: `pseudolabeling loop/merge_labeled_songs.py`)
- User → Training deduplication: runs audit (evidence: `remove_training_duplicates_v3.py`)
- CSV splits → Training deduplication: reads splits (evidence: `remove_training_duplicates_v3.py`)
- Training deduplication → Duplicate reports: writes reports (evidence: `remove_training_duplicates_v3.py`)
- User → Split collision audit: runs audit (evidence: `inspect_split_identity_collisions.py`)
- CSV splits → Split collision audit: reads splits (evidence: `inspect_split_identity_collisions.py`)
- Split collision audit → Collision report: writes report (evidence: `inspect_split_identity_collisions.py`)
- User → Split overlap correction: runs correction (evidence: `remove_validation_test_overlap_fixed.py`)
- CSV splits → Split overlap correction: checks validation and test (evidence: `remove_validation_test_overlap_fixed.py`)
