# bytedance/hllm architecture

> Architecture diagram of the GitHub repository bytedance/hllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bytedance/hllm
- Repository: https://github.com/bytedance/hllm

## Overview

##### Project type and purpose
- This is an open-source machine-learning training and inference pipeline for sequential recommendation. It leverages hierarchical large language models (HLLM) or ID-based nets (IDNet) to model items and users, built on top of PyTorch, DeepSpeed, HuggingFace Transformers, and PyTorch Lightning.

##### File structure
- Top-level “code” folder: contains three sub-modules
- code/REC: core framework (data loaders, models, trainers, evaluators, utilities)
- code/HLLM and code/IDNet: model-specific YAML configs and architecture definitions
- overall/*.yaml: high-level experiment configurations (single-node or deepspeed, LLM vs. IDNet)
- main.py & run.py: CLI entry points that parse args, load configs, and launch training/inference
- dataset/ and information/: raw and processed interaction data and item metadata
- process_books.py: data preprocessing script for Amazon Books
- reproduce/: shell scripts to reproduce benchmark runs
- requirements.txt: external library dependencies

##### README insights
- The system is fully configurable via CLI flags and YAML config files, with clear separation of model hyperparameters (HLLM.yaml, sasrec.yaml, etc.) and experiment settings (overall/*.yaml).
- Training uses DeepSpeed or torchrun for distributed training; evaluation can be toggled with a flag.
- Logging is handled via WandB (optional) and custom logger.

##### Detail without overthinking
- Break the system into as many self-contained components as it has directories/modules
- Every top-level folder or critical YAML is represented
- Both training and inference paths (–val_only flag directs Trainer to skip backward pass)
- Reproducibility scripts (reproduce/*.sh) as an auxiliary component for experiment orchestration

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Scripts & Entry
    subgraph "CLI Entrypoint" 
        direction TB
        Main["main.py"]:::script 
        Run["run.py"]:::script
    end
    subgraph "Reproducibility Scripts"
        direction TB
        Repro1["HLLM-Books-neg28k.sh"]:::script
        Repro2["HLLM-Books-neg512.sh"]:::script
        Repro3["SASRec-Books-neg512.sh"]:::script
    end
    Process["process_books.py"]:::script

    %% Configuration
    subgraph "Configuration Manager"
        direction TB
        ConfigCore["configurator.py"]:::config
        ID_YAML["overall/ID.yaml"]:::config
        ID_DS_YAML["overall/ID_deepspeed.yaml"]:::config
        LLM_DDP["overall/LLM_ddp.yaml"]:::config
        LLM_DS["overall/LLM_deepspeed.yaml"]:::config
        HLLM_YAML["HLLM/HLLM.yaml"]:::config
        HSTU_YAML["IDNet/hstu.yaml"]:::config
        LLAMAID_YAML["IDNet/llama_id.yaml"]:::config
        SASREC_YAML["IDNet/sasrec.yaml"]:::config
    end

    %% Data Ingestion & Processing
    subgraph "Data Ingestion & Processing"
        direction TB
        Dataset["dataset/"]:::data
        Information["information/"]:::data
        Dataload["dataload.py"]:::data
        DataUtils["utils.py"]:::data
        Batchset["batchset.py"]:::data
        Collate["collate_fn.py"]:::data
        Trainset["trainset.py"]:::data
        Evalset["evalset.py"]:::data
    end

    %% Model Factory
    subgraph "Model Factory"
        direction TB
        BaseModel["basemodel.py"]:::compute
        Layers["layers.py"]:::compute
        HLLM_core["hllm.py"]:::compute
        Activations["activations.py"]:::compute
        FlashAttn["flash_self_attn.py"]:::compute
        BERT["modeling_bert.py"]:::compute
        LLAMA["modeling_llama.py"]:::compute
        Mistral["modeling_mistral.py"]:::compute
        RopeUtils["modeling_rope_utils.py"]:::compute
        ConfigBaichuan["configuration_baichuan.py"]:::compute
        ModelBaichuan["modeling_baichuan.py"]:::compute
        HSTU_Model["hstu.py"]:::compute
        SASREC_Model["sasrec.py"]:::compute
        LLAmIDRec["llmidrec.py"]:::compute
    end

    %% Training & Evaluation & Logging
    subgraph "Training & Evaluation"
        direction TB
        Trainer["trainer.py"]:::compute
        Evaluator["evaluator.py"]:::compute
        BaseMetric["base_metric.py"]:::compute
        Metrics["metrics.py"]:::compute
        Collector["collector.py"]:::compute
        Register["register.py"]:::compute
        EvalUtils["utils.py"]:::compute
    end

    subgraph "Logging & Checkpointing"
        direction TB
        Logger["logger.py"]:::compute
        WandBLogger["wandblogger.py"]:::compute
        LRScheduler["lr_scheduler.py"]:::compute
    end

    External["External Services & Libraries"]:::external

    %% Relationships
    Main -->|parses & launches| ConfigCore
    Run -->|parses & launches| ConfigCore
    ConfigCore -->|loads| ID_YAML
    ConfigCore -->|loads| ID_DS_YAML
    ConfigCore -->|loads| LLM_DDP
    ConfigCore -->|loads| LLM_DS
    ConfigCore -->|loads| HLLM_YAML
    ConfigCore -->|loads| HSTU_YAML
    ConfigCore -->|loads| LLAMAID_YAML
    ConfigCore -->|loads| SASREC_YAML

    ConfigCore -->|injects params| Dataload
    ConfigCore -->|injects params| BaseModel

    Process -->|preprocesses data to| Dataset
    Dataset -->|raw data| Dataload
    Information -->|metadata| Dataload
    Dataload -->|batches| Trainer
    DataUtils -->|utils| Trainer
    Batchset -->|batch creation| Trainer
    Collate -->|collation| Trainer
    Trainset -->|train split| Trainer
    Evalset -->|eval split| Trainer

    BaseModel -->|defines base| Trainer
    Layers -->|layers| Trainer
    HLLM_core -->|HLLM modules| Trainer
    Activations -->|activations| Trainer
    FlashAttn -->|efficient attn| Trainer
    BERT -->|BERT model| Trainer
    LLAMA -->|Llama model| Trainer
    Mistral -->|Mistral model| Trainer
    RopeUtils -->|rope utils| Trainer
    ConfigBaichuan -->|Baichuan config| Trainer
    ModelBaichuan -->|Baichuan model| Trainer
    HSTU_Model -->|HSTU model| Trainer
    SASREC_Model -->|SASRec model| Trainer
    LLAmIDRec -->|LLM-ID model| Trainer

    Trainer -->|validation outputs| Evaluator
    Evaluator -->|metrics| Trainer
    BaseMetric -->|metric defs| Evaluator
    Metrics -->|calculates| Evaluator
    Collector -->|collects| Evaluator
    Register -->|registry| Evaluator
    EvalUtils -->|utils| Evaluator

    Trainer -->|logs & saves| Logger
    Trainer -->|logs metrics| WandBLogger
    Trainer -->|schedules LR| LRScheduler

    Trainer -->|uses| External
    BaseModel -->|uses| External
    Dataload -->|uses| External
    Logger -->|optional WandB| External

    Main -->|orchestrates| Repro1
    Main -->|orchestrates| Repro2
    Main -->|orchestrates| Repro3

    %% Click Events
    click Main "https://github.com/bytedance/hllm/blob/main/code/main.py"
    click Run "https://github.com/bytedance/hllm/blob/main/code/run.py"
    click Process "https://github.com/bytedance/hllm/blob/main/process_books.py"
    click Repro1 "https://github.com/bytedance/hllm/blob/main/reproduce/HLLM-Books-neg28k.sh"
    click Repro2 "https://github.com/bytedance/hllm/blob/main/reproduce/HLLM-Books-neg512.sh"
    click Repro3 "https://github.com/bytedance/hllm/blob/main/reproduce/SASRec-Books-neg512.sh"
    click ConfigCore "https://github.com/bytedance/hllm/blob/main/code/REC/config/configurator.py"
    click ID_YAML "https://github.com/bytedance/hllm/blob/main/code/overall/ID.yaml"
    click ID_DS_YAML "https://github.com/bytedance/hllm/blob/main/code/overall/ID_deepspeed.yaml"
    click LLM_DDP "https://github.com/bytedance/hllm/blob/main/code/overall/LLM_ddp.yaml"
    click LLM_DS "https://github.com/bytedance/hllm/blob/main/code/overall/LLM_deepspeed.yaml"
    click HLLM_YAML "https://github.com/bytedance/hllm/blob/main/code/HLLM/HLLM.yaml"
    click HSTU_YAML "https://github.com/bytedance/hllm/blob/main/code/IDNet/hstu.yaml"
    click LLAMAID_YAML "https://github.com/bytedance/hllm/blob/main/code/IDNet/llama_id.yaml"
    click SASREC_YAML "https://github.com/bytedance/hllm/blob/main/code/IDNet/sasrec.yaml"
    click Dataset "https://github.com/bytedance/hllm/blob/main/dataset/readme.md"
    click Information "https://github.com/bytedance/hllm/blob/main/information/readme.md"
    click Dataload "https://github.com/bytedance/hllm/blob/main/code/REC/data/dataload.py"
    click DataUtils "https://github.com/bytedance/hllm/blob/main/code/REC/data/utils.py"
    click Batchset "https://github.com/bytedance/hllm/blob/main/code/REC/data/dataset/batchset.py"
    click Collate "https://github.com/bytedance/hllm/blob/main/code/REC/data/dataset/collate_fn.py"
    click Trainset "https://github.com/bytedance/hllm/blob/main/code/REC/data/dataset/trainset.py"
    click Evalset "https://github.com/bytedance/hllm/blob/main/code/REC/data/dataset/evalset.py"
    click BaseModel "https://github.com/bytedance/hllm/blob/main/code/REC/model/basemodel.py"
    click Layers "https://github.com/bytedance/hllm/blob/main/code/REC/model/layers.py"
    click HLLM_core "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/hllm.py"
    click Activations "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/activations.py"
    click FlashAttn "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/flash_self_attn.py"
    click BERT "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/modeling_bert.py"
    click LLAMA "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/modeling_llama.py"
    click Mistral "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/modeling_mistral.py"
    click RopeUtils "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/modeling_rope_utils.py"
    click ConfigBaichuan "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/baichuan/configuration_baichuan.py"
    click ModelBaichuan "https://github.com/bytedance/hllm/blob/main/code/REC/model/HLLM/baichuan/modeling_baichuan.py"
    click HSTU_Model "https://github.com/bytedance/hllm/blob/main/code/REC/model/IDNet/hstu.py"
    click SASREC_Model "https://github.com/bytedance/hllm/blob/main/code/REC/model/IDNet/sasrec.py"
    click LLAmIDRec "https://github.com/bytedance/hllm/blob/main/code/REC/model/IDNet/llmidrec.py"
    click Trainer "https://github.com/bytedance/hllm/blob/main/code/REC/trainer/trainer.py"
    click Evaluator "https://github.com/bytedance/hllm/blob/main/code/REC/evaluator/evaluator.py"
    click BaseMetric "https://github.com/bytedance/hllm/blob/main/code/REC/evaluator/base_metric.py"
    click Metrics "https://github.com/bytedance/hllm/blob/main/code/REC/evaluator/metrics.py"
    click Collector "https://github.com/bytedance/hllm/blob/main/code/REC/evaluator/collector.py"
    click Register "https://github.com/bytedance/hllm/blob/main/code/REC/evaluator/register.py"
    click EvalUtils "https://github.com/bytedance/hllm/blob/main/code/REC/evaluator/utils.py"
    click Logger "https://github.com/bytedance/hllm/blob/main/code/REC/utils/logger.py"
    click WandBLogger "https://github.com/bytedance/hllm/blob/main/code/REC/utils/wandblogger.py"
    click LRScheduler "https://github.com/bytedance/hllm/blob/main/code/REC/utils/lr_scheduler.py"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef compute fill:#d4edda,stroke:#333,stroke-width:1px
    classDef config fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef external fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef script fill:#f8d7da,stroke:#333,stroke-width:1px
```
