pronob29/pl-ncf-comparison

1. This repository is an offline recommendation-research pipeline, not an application server. Its core loop is: train models on interaction data, extract learned embeddings, score their recommendation and clustering quality, then aggregate results into tables and plots.

2. The source boundary is src/. It contains the model implementations (mf.py, mlp.py, neumf.py), the pseudo-labeling extension (pl.py), and the training/evaluation stack (train_val_test.py, engine.py, metrics.py, data.py, split_utils.py, support_groups_utils.py, config.py, utils.py).

3. There are six model variants in the experiment matrix: MF, MLP, and NeuMF, each with a baseline version and a pseudo-labeling version. The PL variants add a second embedding branch so the model learns both recommendation embeddings and PL-specific embeddings.

4. The pseudo-labeling path is a key architectural boundary. It uses survey-derived user/item alignment features to create pseudo-label targets, typically via cosine similarity. Training combines standard recommendation loss with a PL loss, so semantic structure is learned alongside ranking performance.

5. Two dataset/evaluation regimes drive the pipeline: support_groups_full_164 uses a standard train/validation/test split, while support_groups_full_164_loo uses leave-one-out ranking evaluation. The repository is organized around these two experiment folders under results/.

6. Training is batch-oriented. src/train_val_test.py is the main entry point, and scripts/train_support_groups_only.sh plus scripts/submit_all_jobs.sh indicate SLURM-based orchestration for running many model/seed combinations on GPU nodes. The checked-in models/trained/ directory stores the resulting .model checkpoints.

7. After training, src/extract_embeddings.py reads the saved checkpoints and writes embedding arrays into results/.../embeddings/. Baseline models produce one user and one item embedding set; PL models produce both main embeddings and PL-specific embeddings.

8. The analysis stage is a second pipeline. Shell wrappers in scripts/ run clustering scoring, UMAP generation, metric aggregation, and comparison plotting in sequence. Utilities under scripts/utils/ select optimal cluster counts, compute silhouette scores, aggregate results, and verify plot quality.

9. Clustering evaluation is based on spherical K-means over L2-normalized embeddings, with cosine silhouette scores written to results/.../clustering/. UMAP visualizations are generated from the embeddings and their metadata are saved as JSON alongside the figures in results/.../umap_plots/.

10. Recommendation quality is summarized separately from clustering quality. The pipeline aggregates HR@5, NDCG@5, and AUC across seeds into performance_metrics.csv and performance_metrics_all_seeds.csv, then rolls everything up into results/comprehensive_results/ for cross-experiment comparison.

11. The final outputs are artifacts, not runtime dependencies: trained checkpoints, .npy embeddings, .csv metric tables, .json plot metadata, and comparison figures. This makes the repository suitable for reproducible research and post-hoc analysis, with results/ acting as the main generated-data boundary.

12. In short, the architecture is a two-stage ML workflow: training/inference in src/, and experiment orchestration/analysis in scripts/, with persistent artifacts in models/trained/ and results/ connecting the stages.