# zyang1580/collm architecture

> Architecture diagram of the GitHub repository zyang1580/collm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zyang1580/collm
- Repository: https://github.com/zyang1580/collm

## Overview

CoLLM is a research recommendation system that combines collaborative user/item representations with an LLM. Its principal workflow builds recommendation datasets and prompts, trains first on text-only inputs (LoRA tuning), then tunes the collaborative-information embedding (CIE) mapping with IDs, and evaluates through recommendation tasks and runners. The graph includes the documented Qwen variant and baseline training branch. Source excerpts show the MF/DIN and SASRec entrypoints and core orchestration; several model and dataset internals are indexed but not excerpted, so their detailed call relationships are omitted.

Main entry points: [`train_collm_mf_din.py`](https://github.com/zyang1580/collm/blob/main/train_collm_mf_din.py) (CoLLM training), [`minigpt4/tasks/rec_base_task.py`](https://github.com/zyang1580/collm/blob/main/minigpt4/tasks/rec_base_task.py) (Recommendation task), [`minigpt4/models/minigpt4rec_v2.py`](https://github.com/zyang1580/collm/blob/main/minigpt4/models/minigpt4rec_v2.py) (CoLLM model), [`minigpt4/runners/runner_base_rec.py`](https://github.com/zyang1580/collm/blob/main/minigpt4/runners/runner_base_rec.py) (Recommendation runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Training entrypoints"]
  node_collm_train["CoLLM training"]
  node_sasrec_train["SASRec training"]
  node_baseline_train["Baseline training"]
end

subgraph group_data["Recommendation data"]
  node_builder["Dataset builders"]
  node_pair_builder["Pair builder"]
  node_rec_datasets["Recommendation datasets<br/>[rec_datasets.py]"]
  node_gnn_dataset["GNN dataset<br/>[rec_gnndataset.py]"]
  node_processors["Prompt processors<br/>[rec_processors.py]"]
  node_data_utils["Dataset utilities<br/>[data_utils.py]"]
end

subgraph group_model["Recommendation models"]
  node_collm_model["CoLLM model<br/>[minigpt4rec_v2.py]"]
  node_qwen_model["Qwen CoLLM"]
  node_rec_model["Collaborative model<br/>[rec_model.py]"]
  node_rec_backbones["Rec backbones<br/>[rec_base_models.py]"]
  node_llama["LLM implementation<br/>[modeling_llama.py]"]
end

subgraph group_runtime["Training runtime"]
  node_task["Recommendation task<br/>[rec_base_task.py]"]
  node_base_task["Task training loop<br/>[base_task.py]"]
  node_runner["Recommendation runner<br/>[runner_base_rec.py]"]
  node_base_runner["Runner base<br/>[runner_base.py]"]
  node_config["Configuration<br/>[config.py]"]
  node_registry["Component registry<br/>[registry.py]"]
end

node_operator(("Researcher"))
node_llm_weights["Pretrained LLM weights"]

node_operator -->|"launches"| node_collm_train
node_collm_train -->|"loads config"| node_config
node_collm_train -->|"builds task"| node_task
node_collm_train -->|"starts training"| node_runner
node_sasrec_train -->|"builds task"| node_task
node_sasrec_train -->|"starts training"| node_runner
node_task -->|"extends"| node_base_task
node_task -.->|"builds model"| node_collm_model
node_collm_train -.->|"loads datasets"| node_builder
node_builder -->|"constructs"| node_rec_datasets
node_builder -.->|"configures"| node_processors
node_rec_datasets -.->|"uses utilities"| node_data_utils
node_sasrec_train -->|"loads dataset"| node_gnn_dataset
node_task -->|"prepares samples"| node_data_utils
node_runner -->|"extends"| node_base_runner
node_runner -->|"evaluates with"| node_task
node_collm_train -->|"registers modules"| node_registry
node_task -.->|"selects components"| node_registry
node_collm_model -->|"uses collaborative model"| node_rec_model
node_qwen_model -->|"uses collaborative model"| node_rec_model
node_collm_model -->|"uses LLM"| node_llama
node_qwen_model -->|"uses LLM implementation"| node_llama
node_rec_model -->|"uses backbones"| node_rec_backbones
node_llm_weights -.->|"initializes"| node_collm_model
node_collm_train -->|"tunes LoRA first"| node_collm_model
node_collm_train -->|"then tunes CIE"| node_collm_model
node_operator -.->|"launches baseline"| node_baseline_train

click node_collm_train "https://github.com/zyang1580/collm/blob/main/train_collm_mf_din.py"
click node_sasrec_train "https://github.com/zyang1580/collm/blob/main/train_collm_sasrec.py"
click node_baseline_train "https://github.com/zyang1580/collm/blob/main/baseline_train_lightgcn_ood.py"
click node_builder "https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/builders/rec_base_dataset_builder.py"
click node_pair_builder "https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/builders/rec_pair_builder.py"
click node_rec_datasets "https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/datasets/rec_datasets.py"
click node_gnn_dataset "https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/datasets/rec_gnndataset.py"
click node_processors "https://github.com/zyang1580/collm/blob/main/minigpt4/processors/rec_processors.py"
click node_data_utils "https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/data_utils.py"
click node_task "https://github.com/zyang1580/collm/blob/main/minigpt4/tasks/rec_base_task.py"
click node_base_task "https://github.com/zyang1580/collm/blob/main/minigpt4/tasks/base_task.py"
click node_runner "https://github.com/zyang1580/collm/blob/main/minigpt4/runners/runner_base_rec.py"
click node_base_runner "https://github.com/zyang1580/collm/blob/main/minigpt4/runners/runner_base.py"
click node_config "https://github.com/zyang1580/collm/blob/main/minigpt4/common/config.py"
click node_registry "https://github.com/zyang1580/collm/blob/main/minigpt4/common/registry.py"
click node_collm_model "https://github.com/zyang1580/collm/blob/main/minigpt4/models/minigpt4rec_v2.py"
click node_qwen_model "https://github.com/zyang1580/collm/blob/main/minigpt4/models/minigpt4rec_v2_qwen.py"
click node_rec_model "https://github.com/zyang1580/collm/blob/main/minigpt4/models/rec_model.py"
click node_rec_backbones "https://github.com/zyang1580/collm/blob/main/minigpt4/models/rec_base_models.py"
click node_llama "https://github.com/zyang1580/collm/blob/main/minigpt4/models/modeling_llama.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_collm_train,node_sasrec_train,node_baseline_train toneBlue
class node_builder,node_pair_builder,node_rec_datasets,node_gnn_dataset,node_processors,node_data_utils toneAmber
class node_collm_model,node_qwen_model,node_rec_model,node_rec_backbones,node_llama toneMint
class node_task,node_base_task,node_runner,node_base_runner,node_config,node_registry toneRose
class node_operator,node_llm_weights toneIndigo
```

## Components

### Training entrypoints

- [CoLLM training](https://github.com/zyang1580/collm/blob/main/train_collm_mf_din.py): `train_collm_mf_din.py`
- [SASRec training](https://github.com/zyang1580/collm/blob/main/train_collm_sasrec.py): `train_collm_sasrec.py`
- [Baseline training](https://github.com/zyang1580/collm/blob/main/baseline_train_lightgcn_ood.py): `baseline_train_lightgcn_ood.py`

### Recommendation data

- [Dataset builders](https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/builders/rec_base_dataset_builder.py): `minigpt4/datasets/builders/rec_base_dataset_builder.py`
- [Pair builder](https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/builders/rec_pair_builder.py): `minigpt4/datasets/builders/rec_pair_builder.py`
- [Recommendation datasets](https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/datasets/rec_datasets.py): `minigpt4/datasets/datasets/rec_datasets.py`
- [GNN dataset](https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/datasets/rec_gnndataset.py): `minigpt4/datasets/datasets/rec_gnndataset.py`
- [Prompt processors](https://github.com/zyang1580/collm/blob/main/minigpt4/processors/rec_processors.py): `minigpt4/processors/rec_processors.py`
- [Dataset utilities](https://github.com/zyang1580/collm/blob/main/minigpt4/datasets/data_utils.py): `minigpt4/datasets/data_utils.py`

### Recommendation models

- [CoLLM model](https://github.com/zyang1580/collm/blob/main/minigpt4/models/minigpt4rec_v2.py): `minigpt4/models/minigpt4rec_v2.py`
- [Qwen CoLLM](https://github.com/zyang1580/collm/blob/main/minigpt4/models/minigpt4rec_v2_qwen.py): `minigpt4/models/minigpt4rec_v2_qwen.py`
- [Collaborative model](https://github.com/zyang1580/collm/blob/main/minigpt4/models/rec_model.py): `minigpt4/models/rec_model.py`
- [Rec backbones](https://github.com/zyang1580/collm/blob/main/minigpt4/models/rec_base_models.py): `minigpt4/models/rec_base_models.py`
- [LLM implementation](https://github.com/zyang1580/collm/blob/main/minigpt4/models/modeling_llama.py): `minigpt4/models/modeling_llama.py`

### Training runtime

- [Recommendation task](https://github.com/zyang1580/collm/blob/main/minigpt4/tasks/rec_base_task.py): `minigpt4/tasks/rec_base_task.py`
- [Task training loop](https://github.com/zyang1580/collm/blob/main/minigpt4/tasks/base_task.py): `minigpt4/tasks/base_task.py`
- [Recommendation runner](https://github.com/zyang1580/collm/blob/main/minigpt4/runners/runner_base_rec.py): `minigpt4/runners/runner_base_rec.py`
- [Runner base](https://github.com/zyang1580/collm/blob/main/minigpt4/runners/runner_base.py): `minigpt4/runners/runner_base.py`
- [Configuration](https://github.com/zyang1580/collm/blob/main/minigpt4/common/config.py): `minigpt4/common/config.py`
- [Component registry](https://github.com/zyang1580/collm/blob/main/minigpt4/common/registry.py): `minigpt4/common/registry.py`

### Other components

- **Researcher**
- **Pretrained LLM weights**

## Connections

- Researcher → CoLLM training: launches (evidence: `README.md`)
- CoLLM training → Configuration: loads config (evidence: `train_collm_mf_din.py`)
- CoLLM training → Recommendation task: builds task (evidence: `train_collm_mf_din.py`)
- CoLLM training → Recommendation runner: starts training (evidence: `train_collm_mf_din.py`)
- SASRec training → Recommendation task: builds task (evidence: `train_collm_sasrec.py`)
- SASRec training → Recommendation runner: starts training (evidence: `train_collm_sasrec.py`)
- Recommendation task → Task training loop: extends (evidence: `minigpt4/tasks/rec_base_task.py`)
- Recommendation task → CoLLM model: builds model
- CoLLM training → Dataset builders: loads datasets
- Dataset builders → Recommendation datasets: constructs (evidence: `minigpt4/datasets/builders/rec_pair_builder.py`)
- Dataset builders → Prompt processors: configures
- Recommendation datasets → Dataset utilities: uses utilities
- SASRec training → GNN dataset: loads dataset (evidence: `train_collm_sasrec.py`)
- Recommendation task → Dataset utilities: prepares samples (evidence: `minigpt4/tasks/rec_base_task.py`)
- Recommendation runner → Runner base: extends (evidence: `minigpt4/runners/runner_base_rec.py`)
- Recommendation runner → Recommendation task: evaluates with (evidence: `minigpt4/runners/runner_base_rec.py`)
- CoLLM training → Component registry: registers modules (evidence: `train_collm_mf_din.py`)
- Recommendation task → Component registry: selects components (evidence: `minigpt4/tasks/rec_base_task.py`)
- CoLLM model → Collaborative model: uses collaborative model (evidence: `minigpt4/models/minigpt4rec_v2.py`)
- Qwen CoLLM → Collaborative model: uses collaborative model (evidence: `minigpt4/models/minigpt4rec_v2_qwen.py`)
- CoLLM model → LLM implementation: uses LLM (evidence: `minigpt4/models/minigpt4rec_v2.py`)
- Qwen CoLLM → LLM implementation: uses LLM implementation (evidence: `minigpt4/models/minigpt4rec_v2_qwen.py`)
- Collaborative model → Rec backbones: uses backbones (evidence: `minigpt4/models/rec_model.py`)
- Pretrained LLM weights → CoLLM model: initializes (evidence: `README.md`)
- CoLLM training → CoLLM model: tunes LoRA first (evidence: `README.md`)
- CoLLM training → CoLLM model: then tunes CIE (evidence: `README.md`)
- Researcher → Baseline training: launches baseline
