How dimensionstp/llm-fine-tune-hf works
Project type and purpose
• This is a command-line driven machine-learning pipeline for large-language-model fine-tuning and scaling. It supports SFT (supervised fine-tuning), DPO (direct preference optimization), LoRA/QLoRA quantized training, Deepspeed acceleration, and optional multi-GPU/Slurm integration. It also calls out to HuggingFace Hub for model uploading.
File structure
• Top-level:
• configs/ — Hydra and pipeline configuration files (dataset, training, Deepspeed, trainer, SFT/DPO).
• main.py — entry-point invoking the Hydra-driven pipeline.
• scripts/ — shell scripts to launch preprocessing, training, Slurm jobs, and HF-Hub uploads.
• src/ — core code, subdivided into
– preprocessing/ — data/tokenizer merging, reasoning specialization
– datasets/ — SFT and DPO dataset wrappers
– pipelines/ — generic pipeline abstraction
– trainers/ — custom DPO trainer logic
– scaling/ — “dus” and “wus” scaling algorithms
– postprocessing/ — scripts to upload models to HuggingFace
– utils/ — helper setup functions
• Use of Hydra for configuration, Deepspeed for distributed training, HuggingFace Transformers/PEFT for model fine-tuning, and Slurm for HPC.
README for additional insights
• Describes required .env variables, quick-start commands, optional flags (is_sft, is_quantized, strategy=deepspeed), and upload_user/model_type for Hub integration.
Detail
• Node with the source folder (src/preprocessing, src/trainers, etc.).
• Branching inside the Trainer node for SFT vs. DPO vs. quantization strategy.
• Optional flows (e.g., skip scaling if not required).