# dimensionstp/llm-fine-tune-hf architecture

> Architecture diagram of the GitHub repository dimensionstp/llm-fine-tune-hf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dimensionstp/llm-fine-tune-hf
- Repository: https://github.com/dimensionstp/llm-fine-tune-hf

## Overview

##### Project type and purpose
- This is a command-line driven machine-learning pipeline for large-language-model fine-tuning and scaling. It supports SFT (supervised fine-tuning), DPO (direct preference optimization), LoRA/QLoRA quantized training, Deepspeed acceleration, and optional multi-GPU/Slurm integration. It also calls out to HuggingFace Hub for model uploading.

##### File structure
- Top-level:
- configs/ — Hydra and pipeline configuration files (dataset, training, Deepspeed, trainer, SFT/DPO).
- main.py — entry-point invoking the Hydra-driven pipeline.
- scripts/ — shell scripts to launch preprocessing, training, Slurm jobs, and HF-Hub uploads.
- src/ — core code, subdivided into
– preprocessing/ — data/tokenizer merging, reasoning specialization
– datasets/ — SFT and DPO dataset wrappers
– pipelines/ — generic pipeline abstraction
– trainers/ — custom DPO trainer logic
– scaling/ — “dus” and “wus” scaling algorithms
– postprocessing/ — scripts to upload models to HuggingFace
– utils/ — helper setup functions
- Use of Hydra for configuration, Deepspeed for distributed training, HuggingFace Transformers/PEFT for model fine-tuning, and Slurm for HPC.

##### README for additional insights
- Describes required .env variables, quick-start commands, optional flags (is_sft, is_quantized, strategy=deepspeed), and upload_user/model_type for Hub integration.

##### Detail
- Node with the source folder (`src/preprocessing`, `src/trainers`, etc.).
- Branching inside the Trainer node for SFT vs. DPO vs. quantization strategy.
- Optional flows (e.g., skip scaling if not required).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Orchestration Scripts"
        train_sh["train.sh"]:::orange
        dpo_train_sh["dpo_train.sh"]:::orange
        preprocess_sh["preprocess.sh"]:::orange
        dus_sh["dus.sh"]:::orange
        wus_sh["wus.sh"]:::orange
        slurm_train_sh["slurm_train.sh"]:::orange
        slurm_dpo_train_sh["slurm_dpo_train.sh"]:::orange
        specialize_reasoning_sh["specialize_reasoning.sh"]:::orange
        upload_to_hf_hub_sh["upload_to_hf_hub.sh"]:::orange
        upload_all_to_hf_hub_sh["upload_all_to_hf_hub.sh"]:::orange
    end

    subgraph "Configuration & Entry"
        main_py["main.py"]:::green
        configs["configs/"]:::green
        env[".env files"]:::green
    end

    subgraph "Pipeline Stages"
        subgraph Preprocessing
            merge_model["merge_model.py"]:::blue
            merge_tokenizer["merge_tokenizer.py"]:::blue
            specialize_reasoning_py["specialize_reasoning.py"]:::blue
        end
        subgraph "Dataset Builder"
            sft_dataset["sft_dataset.py"]:::blue
            dpo_dataset["dpo_dataset.py"]:::blue
        end
        pipeline_abstr["pipeline.py"]:::blue
        custom_trainer["custom_dpo_trainer.py"]:::blue
        dus_algo["dus.py"]:::blue
        wus_algo["wus.py"]:::blue
        upload_hf_hub["upload_to_hf_hub.py"]:::blue
        upload_all_hf_hub["upload_all_to_hf_hub.py"]:::blue
    end

    subgraph "External Services"
        hub["HuggingFace Hub"]:::orange
        slurm_cluster["HPC Cluster (Slurm)"]:::orange
    end

    %% Orchestration flow
    train_sh -->|invokes| main_py
    dpo_train_sh -->|invokes| main_py
    preprocess_sh -->|invokes| main_py
    dus_sh -->|invokes| main_py
    wus_sh -->|invokes| main_py
    slurm_train_sh -->|wraps| slurm_cluster
    slurm_dpo_train_sh -->|wraps| slurm_cluster
    slurm_cluster -->|runs| main_py
    specialize_reasoning_sh -->|invokes| main_py
    upload_to_hf_hub_sh -->|invokes| main_py
    upload_all_to_hf_hub_sh -->|invokes| main_py

    %% Configuration flow
    main_py -->|loads YAML| configs
    main_py -->|loads env| env

    %% Pipeline flow
    main_py -->|calls| merge_model
    merge_model --> merge_tokenizer
    merge_tokenizer --> specialize_reasoning_py
    specialize_reasoning_py --> sft_dataset
    specialize_reasoning_py --> dpo_dataset
    sft_dataset --> pipeline_abstr
    dpo_dataset --> pipeline_abstr
    pipeline_abstr --> custom_trainer
    custom_trainer --> dus_algo
    custom_trainer --> wus_algo
    dus_algo --> upload_hf_hub
    wus_algo --> upload_hf_hub
    custom_trainer --> upload_hf_hub
    upload_hf_hub --> upload_all_hf_hub
    upload_all_hf_hub --> hub

    %% Click Events
    click train_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/train.sh"
    click dpo_train_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/dpo_train.sh"
    click preprocess_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/preprocess.sh"
    click dus_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/dus.sh"
    click wus_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/wus.sh"
    click slurm_train_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/slurm_train.sh"
    click slurm_dpo_train_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/slurm_dpo_train.sh"
    click specialize_reasoning_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/specialize_reasoning.sh"
    click upload_to_hf_hub_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/upload_to_hf_hub.sh"
    click upload_all_to_hf_hub_sh "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/scripts/upload_all_to_hf_hub.sh"
    click main_py "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/main.py"
    click configs "https://github.com/dimensionstp/llm-fine-tune-hf/tree/main/configs/"
    click merge_model "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/preprocessing/merge_model.py"
    click merge_tokenizer "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/preprocessing/merge_tokenizer.py"
    click specialize_reasoning_py "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/preprocessing/specialize_reasoning.py"
    click sft_dataset "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/datasets/sft_dataset.py"
    click dpo_dataset "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/datasets/dpo_dataset.py"
    click pipeline_abstr "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/pipelines/pipeline.py"
    click custom_trainer "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/trainers/custom_dpo_trainer.py"
    click dus_algo "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/scaling/dus.py"
    click wus_algo "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/scaling/wus.py"
    click upload_hf_hub "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/postprocessing/upload_to_hf_hub.py"
    click upload_all_hf_hub "https://github.com/dimensionstp/llm-fine-tune-hf/blob/main/src/postprocessing/upload_all_to_hf_hub.py"

    %% Styles
    classDef blue fill:#cce5ff,stroke:#3399ff,color:#000;
    classDef green fill:#d4edda,stroke:#2e7d32,color:#000;
    classDef orange fill:#fff3cd,stroke:#e69138,color:#000;
```
