How bytedance/hllm works
Project type and purpose
• This is an open-source machine-learning training and inference pipeline for sequential recommendation. It leverages hierarchical large language models (HLLM) or ID-based nets (IDNet) to model items and users, built on top of PyTorch, DeepSpeed, HuggingFace Transformers, and PyTorch Lightning.
File structure
• Top-level “code” folder: contains three sub-modules
• code/REC: core framework (data loaders, models, trainers, evaluators, utilities)
• code/HLLM and code/IDNet: model-specific YAML configs and architecture definitions
• overall/*.yaml: high-level experiment configurations (single-node or deepspeed, LLM vs. IDNet)
• main.py & run.py: CLI entry points that parse args, load configs, and launch training/inference
• dataset/ and information/: raw and processed interaction data and item metadata
• process_books.py: data preprocessing script for Amazon Books
• reproduce/: shell scripts to reproduce benchmark runs
• requirements.txt: external library dependencies
README insights
• The system is fully configurable via CLI flags and YAML config files, with clear separation of model hyperparameters (HLLM.yaml, sasrec.yaml, etc.) and experiment settings (overall/*.yaml).
• Training uses DeepSpeed or torchrun for distributed training; evaluation can be toggled with a flag.
• Logging is handled via WandB (optional) and custom logger.
Detail without overthinking
• Break the system into as many self-contained components as it has directories/modules
• Every top-level folder or critical YAML is represented
• Both training and inference paths (–val_only flag directs Trainer to skip backward pass)
• Reproducibility scripts (reproduce/*.sh) as an auxiliary component for experiment orchestration