# hkuds/xrec architecture

> Architecture diagram of the GitHub repository hkuds/xrec, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hkuds/xrec
- Repository: https://github.com/hkuds/xrec

## Overview

##### Project type and purpose
- This is a research‐oriented machine‐learning pipeline for explainable recommendation (XRec). It integrates a graph‐based collaborative filtering encoder (GNN) with a large language model (LLM) explainer to generate human‐readable explanations for user–item recommendations.

##### Key architectural layers (top‐level directories)
- data: raw and preprocessed datasets (Amazon, Google, Yelp)
- process: data‐preprocessing & splitting
- generation: OpenAI API calls to generate user profiles, item profiles, and ground‐truth explanations
- encoder: GNN model (LightGCN) training to produce user/item embeddings and MoE adapter
- explainer: LLaMA‐based fine‐tuning & inference for explanation generation
- evaluation: metrics computation and system prompts for evaluation

##### Main components and their responsibilities
a. Data Ingestion & Storage
- Raw CSV/JSON files under data/<dataset>
- Pickles for embeddings, converters, and train/val/test splits
b. Preprocessing Pipeline (process/)
- data.py, interaction.py, metadata.py, para_dict.py, separate.py
- Cleans raw data, generates interaction matrices, profile dictionaries, and splits into trn/val/tst
c. Profile & Explanation Generation (generation/)
- Three sub‐modules for item_profile, user_profile, explanation
- Each uses OpenAI API with system & user prompts to produce profiles/explanations
- Outputs JSON and pickles under data/<dataset>
d. Collaborative Filtering Encoder (encoder/)
- models/lightgcn.py: LightGCN architecture
- train_encoder.py: trains GNN on processed interactions, outputs embeddings & MoE adapter pickles
- utils: data loading, loss, metrics, parsing
e. LLM Explainer (explainer/)
- modeling_explainer.py: wraps a modified LLaMA model with MoE adapter integration
- explainer.py: defines the XRec model architecture
- main.py: orchestrates finetuning (“--mode finetune”) and generation (“--mode generate”)
- sample.py: prints sample explanations
f. Evaluation Module (evaluation/)
- main.py: runs evaluation over generated explanations vs. ground truth
- metrics.py: computes BLEU, ROUGE, GPTScore, etc.
- system_prompt.txt: prompt for GPTScore

##### External dependencies & services
- OpenAI API (for generation phase)
- Hugging Face Hub (to download Llama 2 weights)
- PyTorch & Transformers

##### Relationships & data flow
1. Raw Data → process/ → processed interactions & metadata
2. Processed Data → generation/ → user_profile.json, item_profile.json, explanation.json
3. Processed Interactions → encoder/ → user_emb.pkl, item_emb.pkl, adapter.pkl
4. {user_profile.json, item_profile.json, user_emb.pkl, item_emb.pkl, adapter.pkl} → explainer/
→ finetune or generate explanations
5. Generated explanations + ground truth → evaluation/ → metrics report

##### Architectural patterns & principles
- Modular pipeline: clear separation between data prep, profile generation, embedding training, language generation, and evaluation
- Model‐agnostic adapter (Mixture-of‐Experts) connecting collaborative signals to LLM
- Micro‐batching of workloads: each directory can run independently
- Use of standardized pickles/JSON for interchange formats

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        Data[(Data Storage)]:::dataClass
    end

    %% Process Layer
    subgraph "Process Layer"
        PP["process/"]:::dataClass
        DP["data.py"]:::dataClass
        IP["interaction.py"]:::dataClass
        MP["metadata.py"]:::dataClass
        PPd["para_dict.py"]:::dataClass
        SP["separate.py"]:::dataClass
    end

    %% Generation Layer
    subgraph "Generation Layer"
        UP["user_profile/"]:::generationClass
        UPP["generate_profile.py"]:::generationClass
        IPF["item_profile/"]:::generationClass
        IPP["generate_profile.py"]:::generationClass
        EXPL["explanation/"]:::generationClass
        EXPG["generate_exp.py"]:::generationClass
    end

    %% Encoder Layer
    subgraph "Encoder Layer"
        EM["encoder/models/"]:::encoderClass
        LE["lightgcn.py"]:::encoderClass
        TE["train_encoder.py"]:::encoderClass
        EU["encoder/utils/"]:::encoderClass
    end

    %% Explainer Layer
    subgraph "Explainer Layer"
        EXMain["main.py"]:::explainerClass
        EXModels["models/"]:::explainerClass
        EXMod["modeling_explainer.py"]:::explainerClass
        EXDef["explainer.py"]:::explainerClass
        EXSamp["sample.py"]:::explainerClass
        EUl["utils/"]:::explainerClass
    end

    %% Evaluation Layer
    subgraph "Evaluation Layer"
        EvalMain["main.py"]:::evaluationClass
        Metrics["metrics.py"]:::evaluationClass
        SysPrompt["system_prompt.txt"]:::evaluationClass
    end

    %% External Services
    subgraph "External Services"
        OpenAI(["Cloud: OpenAI API"]):::externalClass
        HF(["Cloud: Hugging Face Hub"]):::externalClass
    end

    %% Data Flows
    Data -->|"raw CSV/JSON"| PP
    PP -->|"processed interactions & metadata"| UP
    PP -->|"processed interactions & metadata"| IPF
    PP -->|"processed interactions & metadata"| EXPL
    PP -->|"processed interactions"| TE

    UP -->|"user_profile.json"| EXMain
    IPF -->|"item_profile.json"| EXMain
    EXPL -->|"explanation.json"| EvalMain

    TE -->|"user_emb.pkl\nitem_emb.pkl\nadapter.pkl"| EXMain

    TE -->|"embeddings"| EU
    EXMain -->|"generated explanations"| EvalMain

    OpenAI --> UP
    OpenAI --> IPF
    OpenAI --> EXMain
    HF --> EXMain

    %% Click Events - Data Layer
    click Data "https://github.com/hkuds/xrec/tree/main/data/"

    %% Click Events - Process Layer
    click PP "https://github.com/hkuds/xrec/tree/main/process/"
    click DP "https://github.com/hkuds/xrec/blob/main/process/data.py"
    click IP "https://github.com/hkuds/xrec/blob/main/process/interaction.py"
    click MP "https://github.com/hkuds/xrec/blob/main/process/metadata.py"
    click PPd "https://github.com/hkuds/xrec/blob/main/process/para_dict.py"
    click SP "https://github.com/hkuds/xrec/blob/main/process/separate.py"

    %% Click Events - Generation Layer
    click UP "https://github.com/hkuds/xrec/tree/main/generation/user_profile/"
    click UPP "https://github.com/hkuds/xrec/blob/main/generation/user_profile/generate_profile.py"
    click IPF "https://github.com/hkuds/xrec/tree/main/generation/item_profile/"
    click IPP "https://github.com/hkuds/xrec/blob/main/generation/item_profile/generate_profile.py"
    click EXPL "https://github.com/hkuds/xrec/tree/main/generation/explanation/"
    click EXPG "https://github.com/hkuds/xrec/blob/main/generation/explanation/generate_exp.py"

    %% Click Events - Encoder Layer
    click EM "https://github.com/hkuds/xrec/tree/main/encoder/models/"
    click LE "https://github.com/hkuds/xrec/blob/main/encoder/models/lightgcn.py"
    click TE "https://github.com/hkuds/xrec/blob/main/encoder/train_encoder.py"
    click EU "https://github.com/hkuds/xrec/tree/main/encoder/utils/"

    %% Click Events - Explainer Layer
    click EXMain "https://github.com/hkuds/xrec/blob/main/explainer/main.py"
    click EXModels "https://github.com/hkuds/xrec/tree/main/explainer/models/"
    click EXMod "https://github.com/hkuds/xrec/blob/main/explainer/models/modeling_explainer.py"
    click EXDef "https://github.com/hkuds/xrec/blob/main/explainer/models/explainer.py"
    click EXSamp "https://github.com/hkuds/xrec/blob/main/explainer/sample.py"
    click EUl "https://github.com/hkuds/xrec/tree/main/explainer/utils/"

    %% Click Events - Evaluation Layer
    click EvalMain "https://github.com/hkuds/xrec/blob/main/evaluation/main.py"
    click Metrics "https://github.com/hkuds/xrec/blob/main/evaluation/metrics.py"
    click SysPrompt "https://github.com/hkuds/xrec/blob/main/evaluation/system_prompt.txt"

    %% Styles
    classDef dataClass fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef generationClass fill:#FFA500,stroke:#333,stroke-width:1px
    classDef encoderClass fill:#90EE90,stroke:#333,stroke-width:1px
    classDef explainerClass fill:#90EE90,stroke:#333,stroke-width:1px
    classDef evaluationClass fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef externalClass fill:#f0f0f0,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
```
