# antgroup/llmopt architecture

> Architecture diagram of the GitHub repository antgroup/llmopt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/antgroup/llmopt
- Repository: https://github.com/antgroup/llmopt

## Overview

##### Project type and purpose
- LLMOPT is a machine-learning framework for “learning to define and solve” optimization problems using large language models (LLMs).
- It comprises three major phases:
- Data preparation & augmentation
- Model learning (Supervised Fine-Tuning and KTO training)
- Inference & automated evaluation (including self-correction)

##### File structure to derive main components
- config/
- Deepspeed JSON configs for distributed training
- data/
- data/data_augment.py (prompt-based augmentation and classification)
- raw testsets (jsonl under data/testset) & example training sets
- prompts/
- Templates for problem formulation, augmentation, classification, self-correction
- sft/ and kto/
- sft/sft.py & script/run_sft.sh → multi-instruction supervised fine-tuning
- kto/kto.py & script/run_kto.sh → KTO (learning-to-define) training
- inference/
- inference.py, inference_q2f2c.py (various inference pipelines)
- merge_model.py (optional model ensemble/merging)
- utils/
- Helpers for augmentation, GPT-4 calls, solution extraction, templating, solver invocation
- script/
- Entrypoint shells for training workflows

##### High-level system components (to represent as distinct blocks)
A. Data Layer (blue)
1. Raw Datasets (data/testset/*.jsonl)
2. Example Training Data (data/trainset_example)
B. Data Processing & Augmentation (blue/green)
- data/data_augment.py → Generate definition prompts
- data/data_classification.py → Filter/classify problems
- prompts/*.py → Prompt templates for all modules
C. Model Learning (green)
1. Supervised Fine-Tuning (SFT)
– Module: sft/sft.py
– Entry: script/run_sft.sh
– Config: config/ds_config_zero3.json
2. Knowledge-to-Optimization (KTO) Training
– Module: kto/kto.py
– Entry: script/run_kto.sh
– Config: same Deepspeed config
D. Model Storage & Registry (purple)
- Hugging Face Hub (pre-trained base & LLMOPT fine-tuned models)
E. Inference & Self-Correction (orange/green)
- inference/inference.py (single-pass)
- inference/inference_q2f2c.py (question→formulation→code)
- inference/merge_model.py (ensemble or checkpoint merging)
F. Evaluation & Metrics (red)
- utils/solve.py → runs generated solver code to compute objective
- Reports Execution Rate (ER), Solving Accuracy (SA), Average Self-Corrections (AST)

- Raw Datasets → Data Processing & Augmentation → Formulated Prompt Examples
- Formulated Prompt Examples → SFT & KTO Training (via torchrun + Deepspeed) → Fine-tuned Models
- Fine-tuned Models ↔ Hugging Face Hub (push/pull)
- Fine-tuned Models + Prompt Templates + utils/gpt4_infer → Inference Pipeline
- Inference Pipeline → Generated Solver Code → utils/solve.py → Results
- Results + Self-Correction Loop (utils/gpt4_util + prompts/self_correction_prompt.py) → Improved Code
- Final Results → Evaluation Metrics Dashboard (TensorBoard logs)

##### Architectural patterns & design principles
- Modular pipeline: clearly separate Data → Training → Inference → Evaluation
- Reusable prompt templates: all prompts live in prompts/
- Config-driven training: Deepspeed JSON + shell scripts orchestrate distributed runs
- Self-correction feedback loop at inference time
- External-service integration: Hugging Face Model Hub, GPT-4 API

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        rawData["Raw Datasets\n(data/testset/*.jsonl)"]:::data
        exampleData["Example Training Data\n(data/trainset_example/)"]:::data
    end

    %% Data Processing & Augmentation
    subgraph "Data Processing & Augmentation"
        augment["data/data_augment.py"]:::compute
        classify["data/data_classification.py"]:::compute
        prompts["prompts/"]:::prompt
    end

    %% Model Learning
    subgraph "Model Learning"
        subgraph "Supervised Fine-Tuning (SFT)"
            sftModule["sft/sft.py"]:::compute
            sftScript["script/run_sft.sh"]:::config
            dsConfig1["config/ds_config_zero3.json"]:::config
        end
        subgraph "Knowledge-to-Optimization (KTO)"
            ktoModule["kto/kto.py"]:::compute
            ktoScript["script/run_kto.sh"]:::config
            dsConfig2["config/ds_config_zero3.json"]:::config
        end
    end

    %% Model Storage & Registry
    modelHub["Hugging Face Hub"]:::external

    %% Inference & Self-Correction
    subgraph "Inference & Self-Correction"
        inference1["inference/inference.py"]:::compute
        inference2["inference/inference_q2f2c.py"]:::compute
        mergeModel["inference/merge_model.py"]:::compute
        selfCorrPrompt["prompts/self_correction_prompt.py"]:::prompt
        gpt4Infer["utils/gpt4_infer.py"]:::compute
        gpt4Util["utils/gpt4_util.py"]:::compute
    end

    %% Evaluation & Metrics
    subgraph "Evaluation & Metrics"
        solverExec["utils/solve.py"]:::eval
        tbLogs["TensorBoard\nLogs"]:::eval
    end

    %% Utility Modules
    subgraph "Utilities"
        templateUtil["utils/template.py"]:::compute
        augmentUtil["utils/augment.py"]:::compute
        extractUtil["utils/extract.py"]:::compute
    end

    %% Data Flows
    rawData --> augment
    rawData --> classify
    augment --> prompts
    classify --> prompts
    prompts --> sftModule
    prompts --> ktoModule

    %% Training Flows
    sftModule --> dsConfig1
    sftScript -.-> sftModule
    ktoModule --> dsConfig2
    ktoScript -.-> ktoModule

    dsConfig1 -->|"orchestrates"| sftModule
    dsConfig2 -->|"orchestrates"| ktoModule

    sftModule -->|push/pull| modelHub
    ktoModule -->|push/pull| modelHub

    %% Inference Flow
    modelHub --> inference1
    modelHub --> inference2
    prompts --> inference2
    gpt4Infer --> inference1
    gpt4Infer --> inference2
    inference1 --> mergeModel
    inference2 --> mergeModel

    %% Self-Correction Loop
    mergeModel --> solverExec
    mergeModel --> gpt4Util
    gpt4Util --> selfCorrPrompt
    selfCorrPrompt --> mergeModel

    %% Evaluation
    solverExec --> tbLogs

    %% Utility Integration
    prompts --> templateUtil
    augment --> augmentUtil
    extractUtil --> inference2

    %% Click Events
    click rawData "https://github.com/antgroup/llmopt/tree/main/data/testset/"
    click exampleData "https://github.com/antgroup/llmopt/tree/main/data/trainset_example/"
    click augment "https://github.com/antgroup/llmopt/blob/main/data/data_augment.py"
    click classify "https://github.com/antgroup/llmopt/blob/main/data/data_classification.py"
    click prompts "https://github.com/antgroup/llmopt/tree/main/prompts/"
    click sftModule "https://github.com/antgroup/llmopt/blob/main/sft/sft.py"
    click sftScript "https://github.com/antgroup/llmopt/blob/main/script/run_sft.sh"
    click ktoModule "https://github.com/antgroup/llmopt/blob/main/kto/kto.py"
    click ktoScript "https://github.com/antgroup/llmopt/blob/main/script/run_kto.sh"
    click dsConfig1 "https://github.com/antgroup/llmopt/blob/main/config/ds_config_zero3.json"
    click inference1 "https://github.com/antgroup/llmopt/blob/main/inference/inference.py"
    click inference2 "https://github.com/antgroup/llmopt/blob/main/inference/inference_q2f2c.py"
    click mergeModel "https://github.com/antgroup/llmopt/blob/main/inference/merge_model.py"
    click selfCorrPrompt "https://github.com/antgroup/llmopt/blob/main/prompts/self_correction_prompt.py"
    click gpt4Infer "https://github.com/antgroup/llmopt/blob/main/utils/gpt4_infer.py"
    click gpt4Util "https://github.com/antgroup/llmopt/blob/main/utils/gpt4_util.py"
    click solverExec "https://github.com/antgroup/llmopt/blob/main/utils/solve.py"
    click templateUtil "https://github.com/antgroup/llmopt/blob/main/utils/template.py"
    click augmentUtil "https://github.com/antgroup/llmopt/blob/main/utils/augment.py"
    click extractUtil "https://github.com/antgroup/llmopt/blob/main/utils/extract.py"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef compute fill:#d4edda,stroke:#333,stroke-width:1px
    classDef external fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef config fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef eval fill:#f8d7da,stroke:#333,stroke-width:1px
    classDef prompt fill:#d1ecf1,stroke:#333,stroke-width:1px
```
