# liushiliushi/conftuner architecture

> Architecture diagram of the GitHub repository liushiliushi/conftuner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/liushiliushi/conftuner
- Repository: https://github.com/liushiliushi/conftuner

## Overview

##### Project Type and Purpose
- This is a machine-learning research codebase (ConfTuner) for fine-tuning large language models (LLMs) to verbalize their confidence. It provides:
- Data generation scripts that query base LLMs (Llama-3.1, Qwen, Mistral, GPT-4o) to produce question+answer+confidence data.
- A training pipeline (uncertainty_sft.py) built on HuggingFace Accelerate, DeepSpeed, and PEFT that applies a custom Tokenized Brier Score loss.
- Inference and evaluation scripts (inference.py, inference_gpt.py) to measure accuracy and calibration, optionally logging to Weights & Biases.
- A rich configuration hierarchy under src/llama_recipes (datasets, inference, training, quantization, etc.).

##### File Structure Analysis
- Top-level
- generate_response.py, inference.py, inference_gpt.py, uncertainty_sft.py: entry-point scripts
- scripts.sh, train_inference.sh: helper shell wrappers
- downloaded_config/adapter_config.json: adapter/hyperparameter defaults
- images/: documentation figures
- src/llama_recipes/
- configs/: JSON/YAML configs for DeepSpeed, PEFT, training, quantization, datasets, wandb
- data/: dataset loaders, samplers, formatters (including llama_guard submodule)
- inference/: model loading, prompt formatting, safety checks
- model_checkpointing/: checkpoint save/load handlers
- policies/: precision, optimizer, checkpoint-wrapping rules
- tools/: weight conversion and comparison utilities
- trained_calibration/: RL-based calibration scripts (DPO, reward models, evaluation)
- utils/: metrics computation, config parsing, FLOP counting, etc.

##### Key README Insights
- Two-stage method: (1) extract probability distribution over confidence tokens, (2) fine-tune with Tokenized Brier Score
- Dependencies: HuggingFace Accelerate, DeepSpeed, PEFT (LoRA/QLoRA), OpenAI API for GPT-4o evaluation, WandB
- Example commands for data generation, fine-tuning on different base models, and inference

a. Main Components
- Data Generation Module
- Training Module
- Inference/Evaluation Module
- Configuration Store
- Model Checkpoint Storage
- External Services: OpenAI API, Weights & Biases
- Compute Infrastructure: GPU cluster via Accelerate + DeepSpeed
b. Interactions and Data Flow
1. Data Generation
– generate_response.py reads Config → queries LLM (via HuggingFace or OpenAI API) → produces JSONL datasets
2. Training
– uncertainty_sft.py loads dataset + configs (DeepSpeed, PEFT, training hyperparams) → launches Accelerate/DeepSpeed on GPUs → writes checkpoints to storage → logs metrics to WandB
3. Inference/Evaluation
– inference.py / inference_gpt.py load checkpoint + dataset → run batched evaluation through LLM → calculate calibration metrics → log to WandB or stdout
c. Architectural Patterns
- Modular recipe-based design (llama_recipes) to separate concerns: data, configs, inference, policies, utils
- Configuration-driven pipeline: JSON/YAML files control DeepSpeed, PEFT, dataset sampling
- Wrapper scripts (shell and Python) for reproducible commands
d. Technologies & Frameworks
- Python 3.x, HuggingFace Accelerate, DeepSpeed, PEFT (LoRA/QLoRA)
- OpenAI SDK for GPT-4o
- Weights & Biases for experiment tracking
- JSON/YAML for configurations
- Shell scripts for orchestration

##### Summary
- Break the system into Data Generation, Training, Inference, Config Management, Storage, and External Services.
- How JSON/YAML configs drive each stage.
- Compute orchestration via Accelerate/DeepSpeed and experiment tracking via WandB.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Config Store
    subgraph "Configuration Store"
        direction TB
        DSConfig["ds_config.json"]:::config
        PEFTConfig["peft.py"]:::config
        TrainConfig["training.py"]:::config
        QuantConfig["quantization.py"]:::config
        DataConfig["datasets.py"]:::config
        WandBConfig["wandb.py"]:::config
    end

    %% Data Generation Stage
    subgraph "Stage 1: Data Generation"
        direction TB
        DataGen["Data Generation Module"]:::python
        DatasetStorage["Generated Dataset"]:::storage
    end

    %% Training Stage
    subgraph "Stage 2: Fine-Tuning"
        direction TB
        Trainer["Training Module (Uncertainty SFT)"]:::python
        CheckpointStorage["Model Checkpoint Storage"]:::storage
    end

    %% Inference Stage
    subgraph "Stage 3: Inference & Evaluation"
        direction TB
        InferencePy["Inference Module (Python)"]:::python
        InferenceGPT["Inference GPT Module"]:::python
        InferenceSh["Inference Shell Wrapper"]:::python
        EvalResults["Evaluation Metrics"]:::storage
    end

    %% External Services
    OpenAI["OpenAI API"]:::external
    WandB["Weights & Biases"]:::external

    %% Data Generation Flows
    DSConfig -->|loads configs| DataGen
    DataGen -->|queries LLM| OpenAI
    DataGen -->|outputs JSONL| DatasetStorage

    %% Training Flows
    DSConfig -->|loads configs| Trainer
    DataConfig -->|dataset params| Trainer
    Trainer -->|launches Accelerate/DeepSpeed| CheckpointStorage
    Trainer -->|logs metrics| WandB
    OpenAI -->|optional GPT-4o eval| Trainer

    %% Inference Flows
    DSConfig -->|loads configs| InferencePy
    TrainConfig -->|hyperparams| InferencePy
    CheckpointStorage -->|loads model| InferencePy
    InferencePy -->|eval| EvalResults
    InferenceGPT -->|GPT evaluation| OpenAI
    InferenceSh --> InferencePy
    InferencePy -->|logs| WandB

    %% Click Events
    click DSConfig "https://github.com/liushiliushi/conftuner/blob/main/src/llama_recipes/configs/ds_config.json"
    click PEFTConfig "https://github.com/liushiliushi/conftuner/blob/main/src/llama_recipes/configs/peft.py"
    click TrainConfig "https://github.com/liushiliushi/conftuner/blob/main/src/llama_recipes/configs/training.py"
    click QuantConfig "https://github.com/liushiliushi/conftuner/blob/main/src/llama_recipes/configs/quantization.py"
    click DataConfig "https://github.com/liushiliushi/conftuner/blob/main/src/llama_recipes/configs/datasets.py"
    click WandBConfig "https://github.com/liushiliushi/conftuner/blob/main/src/llama_recipes/configs/wandb.py"
    click DataGen "https://github.com/liushiliushi/conftuner/blob/main/src/generate_response.py"
    click Trainer "https://github.com/liushiliushi/conftuner/blob/main/src/uncertainty_sft.py"
    click InferencePy "https://github.com/liushiliushi/conftuner/blob/main/src/inference.py"
    click InferenceGPT "https://github.com/liushiliushi/conftuner/blob/main/src/inference_gpt.py"
    click InferenceSh "https://github.com/liushiliushi/conftuner/blob/main/src/inference.sh"

    %% Styles
    classDef python fill:#D0E6FF,stroke:#0366d6,color:#0366d6
    classDef config fill:#E6FFDC,stroke:#28A745,color:#28A745
    classDef external fill:#FFEFD5,stroke:#FF8C00,color:#FF8C00
    classDef storage fill:#F0F0F0,stroke:#696969,color:#696969
```
