# pronob29/pl-ncf-comparison architecture

> Architecture diagram of the GitHub repository pronob29/pl-ncf-comparison, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pronob29/pl-ncf-comparison
- Repository: https://github.com/pronob29/pl-ncf-comparison

## Overview

1. This repository is an offline recommendation-research pipeline, not an application server. Its core loop is: train models on interaction data, extract learned embeddings, score their recommendation and clustering quality, then aggregate results into tables and plots.

2. The source boundary is `src/`. It contains the model implementations (`mf.py`, `mlp.py`, `neumf.py`), the pseudo-labeling extension (`pl.py`), and the training/evaluation stack (`train_val_test.py`, `engine.py`, `metrics.py`, `data.py`, `split_utils.py`, `support_groups_utils.py`, `config.py`, `utils.py`).

3. There are six model variants in the experiment matrix: MF, MLP, and NeuMF, each with a baseline version and a pseudo-labeling version. The PL variants add a second embedding branch so the model learns both recommendation embeddings and PL-specific embeddings.

4. The pseudo-labeling path is a key architectural boundary. It uses survey-derived user/item alignment features to create pseudo-label targets, typically via cosine similarity. Training combines standard recommendation loss with a PL loss, so semantic structure is learned alongside ranking performance.

5. Two dataset/evaluation regimes drive the pipeline: `support_groups_full_164` uses a standard train/validation/test split, while `support_groups_full_164_loo` uses leave-one-out ranking evaluation. The repository is organized around these two experiment folders under `results/`.

6. Training is batch-oriented. `src/train_val_test.py` is the main entry point, and `scripts/train_support_groups_only.sh` plus `scripts/submit_all_jobs.sh` indicate SLURM-based orchestration for running many model/seed combinations on GPU nodes. The checked-in `models/trained/` directory stores the resulting `.model` checkpoints.

7. After training, `src/extract_embeddings.py` reads the saved checkpoints and writes embedding arrays into `results/.../embeddings/`. Baseline models produce one user and one item embedding set; PL models produce both main embeddings and PL-specific embeddings.

8. The analysis stage is a second pipeline. Shell wrappers in `scripts/` run clustering scoring, UMAP generation, metric aggregation, and comparison plotting in sequence. Utilities under `scripts/utils/` select optimal cluster counts, compute silhouette scores, aggregate results, and verify plot quality.

9. Clustering evaluation is based on spherical K-means over L2-normalized embeddings, with cosine silhouette scores written to `results/.../clustering/`. UMAP visualizations are generated from the embeddings and their metadata are saved as JSON alongside the figures in `results/.../umap_plots/`.

10. Recommendation quality is summarized separately from clustering quality. The pipeline aggregates HR@5, NDCG@5, and AUC across seeds into `performance_metrics.csv` and `performance_metrics_all_seeds.csv`, then rolls everything up into `results/comprehensive_results/` for cross-experiment comparison.

11. The final outputs are artifacts, not runtime dependencies: trained checkpoints, `.npy` embeddings, `.csv` metric tables, `.json` plot metadata, and comparison figures. This makes the repository suitable for reproducible research and post-hoc analysis, with `results/` acting as the main generated-data boundary.

12. In short, the architecture is a two-stage ML workflow: training/inference in `src/`, and experiment orchestration/analysis in `scripts/`, with persistent artifacts in `models/trained/` and `results/` connecting the stages.

Main entry points: [`src/engine.py`](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/engine.py) (Engine), [`results/support_groups_full_164/embeddings`](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/embeddings) (Embeddings), [`results/support_groups_full_164`](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164) (support_groups_full_164), [`src/train_val_test.py`](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/train_val_test.py) (Train/Test)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_src["src"]
  node_src_train["Train/Test<br/>entrypoint<br/>[train_val_test.py]"]
  node_src_engine["Engine<br/>trainer<br/>[engine.py]"]
  node_src_data["Data Prep<br/>data pipeline<br/>[data.py]"]
  node_src_split["Splits<br/>split logic<br/>[split_utils.py]"]
  node_src_support["Support Groups<br/>feature builder"]
  node_src_config["Config<br/>[config.py]"]
  node_src_models["MF / MLP / NeuMF<br/>recommender models<br/>[mf.py]"]
  node_src_pl["Pseudo-labeling<br/>auxiliary branch<br/>[pl.py]"]
  node_src_metrics["Metrics<br/>ranking + clustering<br/>[metrics.py]"]
  node_src_extract["Extract Emb<br/>postprocess"]
  node_src_umap["UMAP Plots<br/>visualization"]
  node_src_compare["Compare Plots<br/>reporting"]
end

subgraph group_scripts["scripts"]
  node_scripts_submit["Submit All<br/>slurm orchestration<br/>[submit_all_jobs.sh]"]
  node_scripts_train["Train Jobs<br/>slurm wrapper"]
  node_scripts_all_analysis["Run Analysis<br/>pipeline wrapper"]
  node_scripts_utils["Analysis Utils<br/>utility scripts"]
  node_scripts_embed["1 Embeddings<br/>analysis step"]
  node_scripts_cluster["2 Clustering<br/>analysis step"]
  node_scripts_umap["3 UMAP<br/>analysis step"]
  node_scripts_perf["4 Metrics<br/>analysis step"]
  node_scripts_plots["5 Plots<br/>analysis step"]
  node_scripts_compcsv["6 Rollup CSV<br/>analysis step"]
end

subgraph group_artifacts["artifacts"]
  node_art_models[("Trained Models<br/>checkpoint store")]
  node_results_main_root[("support_groups_full_164<br/>experiment outputs")]
  node_results_loo_root[("support_groups_full_164_loo<br/>experiment outputs")]
  node_results_comp[("Comprehensive<br/>summary tables")]
  node_res_embeddings[("Embeddings<br/>artifact set")]
  node_res_clustering[("Clustering<br/>quality scores")]
  node_res_umap[("UMAP Plots<br/>visual outputs")]
  node_res_perf["Perf Tables<br/>metrics tables"]
end

node_scripts_submit -->|"launches"| node_scripts_train
node_scripts_train -->|"runs"| node_src_train
node_src_train -->|"reads"| node_src_config
node_src_train -->|"uses"| node_src_data
node_src_data -->|"splits"| node_src_split
node_src_data -->|"features"| node_src_support
node_src_train -->|"drives"| node_src_engine
node_src_engine -->|"trains"| node_src_models
node_src_engine -->|"augments"| node_src_pl
node_src_engine -->|"scores"| node_src_metrics
node_src_engine -->|"saves"| node_art_models
node_src_extract -->|"loads"| node_art_models
node_src_extract -->|"writes"| node_res_embeddings
node_scripts_embed -->|"wraps"| node_src_extract
node_scripts_cluster -.->|"uses"| node_scripts_utils
node_scripts_umap -->|"wraps"| node_src_umap
node_scripts_perf -.->|"uses"| node_scripts_utils
node_scripts_plots -->|"wraps"| node_src_compare
node_scripts_compcsv -.->|"uses"| node_scripts_utils
node_src_umap -->|"reads"| node_res_embeddings
node_src_umap -->|"writes"| node_res_umap
node_scripts_cluster -->|"reads"| node_res_embeddings
node_scripts_cluster -->|"writes"| node_res_clustering
node_scripts_perf -->|"writes"| node_res_perf
node_src_compare -->|"reads"| node_res_perf
node_src_compare -->|"writes"| node_results_comp
node_results_main_root -->|"contains"| node_res_embeddings
node_results_main_root -->|"contains"| node_res_clustering
node_results_main_root -->|"contains"| node_res_umap
node_results_main_root -->|"contains"| node_res_perf
node_results_loo_root -.->|"mirrors"| node_res_embeddings

click node_src_train "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/train_val_test.py"
click node_src_engine "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/engine.py"
click node_src_data "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/data.py"
click node_src_split "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/split_utils.py"
click node_src_support "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/support_groups_utils.py"
click node_src_config "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/config.py"
click node_src_models "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/mf.py"
click node_src_pl "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/pl.py"
click node_src_metrics "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/metrics.py"
click node_src_extract "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/extract_embeddings.py"
click node_src_umap "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/generate_umap_plots.py"
click node_src_compare "https://github.com/pronob29/pl-ncf-comparison/blob/main/src/generate_comparison_plots.py"
click node_scripts_submit "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/submit_all_jobs.sh"
click node_scripts_train "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/train_support_groups_only.sh"
click node_scripts_all_analysis "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/run_all_analysis_optimal.sh"
click node_scripts_utils "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/utils/compute_silhouette_scores.py"
click node_scripts_embed "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/1_extract_embeddings.sh"
click node_scripts_cluster "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/2_compute_clustering_metrics.sh"
click node_scripts_umap "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/3_generate_umap_plots_all_optimal.sh"
click node_scripts_perf "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/4_aggregate_performance_metrics.sh"
click node_scripts_plots "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/5_generate_comparison_plots.sh"
click node_scripts_compcsv "https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/6_create_comprehensive_csv.sh"
click node_art_models "https://github.com/pronob29/pl-ncf-comparison/tree/main/models/trained"
click node_results_main_root "https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164"
click node_results_loo_root "https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164_loo"
click node_results_comp "https://github.com/pronob29/pl-ncf-comparison/tree/main/results/comprehensive_results"
click node_res_embeddings "https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/embeddings"
click node_res_clustering "https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/clustering"
click node_res_umap "https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/umap_plots"
click node_res_perf "https://github.com/pronob29/pl-ncf-comparison/blob/main/results/support_groups_full_164/performance_metrics.csv"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_train,node_src_engine,node_src_data,node_src_split,node_src_support,node_src_config,node_src_models,node_src_pl,node_src_metrics,node_src_extract,node_src_umap,node_src_compare toneBlue
class node_scripts_submit,node_scripts_train,node_scripts_all_analysis,node_scripts_utils,node_scripts_embed,node_scripts_cluster,node_scripts_umap,node_scripts_perf,node_scripts_plots,node_scripts_compcsv toneAmber
class node_art_models,node_results_main_root,node_results_loo_root,node_results_comp,node_res_embeddings,node_res_clustering,node_res_umap,node_res_perf toneMint
```

## Components

### src

Core training, model, and analysis code

- [Train/Test](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/train_val_test.py): entrypoint · Main training and evaluation loop · `src/train_val_test.py`
- [Engine](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/engine.py): trainer · Batch training, validation, and test execution · `src/engine.py`
- [Data Prep](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/data.py): data pipeline · Loads interactions and prepares train/eval data · `src/data.py`
- [Splits](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/split_utils.py): split logic · Standard and leave-one-out partitioning · `src/split_utils.py`
- [Support Groups](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/support_groups_utils.py): feature builder · Survey-derived alignment and grouping features · `src/support_groups_utils.py`
- [Config](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/config.py): run config · Experiment settings and model selection · `src/config.py`
- [MF / MLP / NeuMF](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/mf.py): recommender models · Baseline recommender implementations · `src/mf.py`
- [Pseudo-labeling](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/pl.py): auxiliary branch · PL loss and second embedding branch · `src/pl.py`
- [Metrics](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/metrics.py): ranking + clustering · Recommendation and embedding quality measures · `src/metrics.py`
- [Extract Emb](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/extract_embeddings.py): postprocess · Loads checkpoints and exports embeddings · `src/extract_embeddings.py`
- [UMAP Plots](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/generate_umap_plots.py): visualization · Builds embedding visualizations and metadata · `src/generate_umap_plots.py`
- [Compare Plots](https://github.com/pronob29/pl-ncf-comparison/blob/main/src/generate_comparison_plots.py): reporting · Creates cross-experiment comparison figures · `src/generate_comparison_plots.py`

### scripts

Batch orchestration and analysis wrappers

- [Submit All](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/submit_all_jobs.sh): slurm orchestration · Top-level job submission for training runs · `scripts/submit_all_jobs.sh`
- [Train Jobs](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/train_support_groups_only.sh): slurm wrapper · Launches model and seed combinations on GPU nodes · `scripts/train_support_groups_only.sh`
- [Run Analysis](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/run_all_analysis_optimal.sh): pipeline wrapper · Executes the post-training analysis sequence · `scripts/run_all_analysis_optimal.sh`
- [Analysis Utils](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/utils/compute_silhouette_scores.py): utility scripts · Clustering, aggregation, and plot-quality helpers · `scripts/utils/compute_silhouette_scores.py`
- [1 Embeddings](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/1_extract_embeddings.sh): analysis step · Wrapper for embedding extraction · `scripts/1_extract_embeddings.sh`
- [2 Clustering](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/2_compute_clustering_metrics.sh): analysis step · Wrapper for clustering evaluation · `scripts/2_compute_clustering_metrics.sh`
- [3 UMAP](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/3_generate_umap_plots_all_optimal.sh): analysis step · Wrapper for UMAP generation over optimal clusters · `scripts/3_generate_umap_plots_all_optimal.sh`
- [4 Metrics](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/4_aggregate_performance_metrics.sh): analysis step · Wrapper for aggregating recommendation metrics · `scripts/4_aggregate_performance_metrics.sh`
- [5 Plots](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/5_generate_comparison_plots.sh): analysis step · Wrapper for comparison plotting · `scripts/5_generate_comparison_plots.sh`
- [6 Rollup CSV](https://github.com/pronob29/pl-ncf-comparison/blob/main/scripts/6_create_comprehensive_csv.sh): analysis step · Wrapper for comprehensive result table creation · `scripts/6_create_comprehensive_csv.sh`

### artifacts

Generated checkpoints and experiment outputs

- [Trained Models](https://github.com/pronob29/pl-ncf-comparison/tree/main/models/trained): checkpoint store · Saved .model checkpoints · `models/trained`
- [support_groups_full_164](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164): experiment outputs · Standard split experiment outputs · `results/support_groups_full_164`
- [support_groups_full_164_loo](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164_loo): experiment outputs · Leave-one-out experiment outputs · `results/support_groups_full_164_loo`
- [Comprehensive](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/comprehensive_results): summary tables · Cross-experiment rollups and summaries · `results/comprehensive_results`
- [Embeddings](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/embeddings): artifact set · User/item and PL-specific embedding arrays · `results/support_groups_full_164/embeddings`
- [Clustering](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/clustering): quality scores · Silhouette scoring outputs for main and PL embeddings · `results/support_groups_full_164/clustering`
- [UMAP Plots](https://github.com/pronob29/pl-ncf-comparison/tree/main/results/support_groups_full_164/umap_plots): visual outputs · UMAP figures and JSON metadata · `results/support_groups_full_164/umap_plots`
- [Perf Tables](https://github.com/pronob29/pl-ncf-comparison/blob/main/results/support_groups_full_164/performance_metrics.csv): metrics tables · HR@5, NDCG@5, and AUC aggregated across seeds · `results/support_groups_full_164/performance_metrics.csv`

## Connections

- Submit All → Train Jobs: launches · Submits the training matrix
- Train Jobs → Train/Test: runs · Invokes the main training entrypoint
- Train/Test → Config: reads · Loads model and dataset configuration
- Train/Test → Data Prep: uses · Builds training and evaluation data
- Data Prep → Splits: splits · Creates standard or leave-one-out partitions
- Data Prep → Support Groups: features · Uses support-group alignment features
- Train/Test → Engine: drives · Delegates batch training and validation
- Engine → MF / MLP / NeuMF: trains · Optimizes MF, MLP, or NeuMF variants
- Engine → Pseudo-labeling: augments · Adds pseudo-label loss and PL embedding branch
- Engine → Metrics: scores · Computes recommendation quality during validation
- Engine → Trained Models: saves · Writes trained checkpoints
- Extract Emb → Trained Models: loads · Reads saved checkpoints for embedding export
- Extract Emb → Embeddings: writes · Stores main and PL-specific embeddings
- 1 Embeddings → Extract Emb: wraps · Batch wrapper for extraction stage
- 2 Clustering → Analysis Utils: uses · Calls clustering utilities
- 3 UMAP → UMAP Plots: wraps · Runs UMAP generation
- 4 Metrics → Analysis Utils: uses · Aggregates recommendation metrics
- 5 Plots → Compare Plots: wraps · Generates comparison figures
- 6 Rollup CSV → Analysis Utils: uses · Builds comprehensive summary CSVs
- UMAP Plots → Embeddings: reads · Consumes exported embeddings
- UMAP Plots → UMAP Plots: writes · Saves figures and plot metadata
- 2 Clustering → Embeddings: reads · Scores main and PL embeddings
- 2 Clustering → Clustering: writes · Stores cosine silhouette results
- 4 Metrics → Perf Tables: writes · Aggregates seed-level recommendation metrics
- Compare Plots → Perf Tables: reads · Compares experiment-level metrics
- Compare Plots → Comprehensive: writes · Produces cross-experiment summaries
- support_groups_full_164 → Embeddings: contains · Standard-split generated embeddings
- support_groups_full_164 → Clustering: contains · Standard-split clustering outputs
- support_groups_full_164 → UMAP Plots: contains · Standard-split UMAP outputs
- support_groups_full_164 → Perf Tables: contains · Standard-split performance tables
- support_groups_full_164_loo → Embeddings: mirrors · Same artifact structure for leave-one-out regime