How antgroup/llmopt works
Project type and purpose
• LLMOPT is a machine-learning framework for “learning to define and solve” optimization problems using large language models (LLMs).
• It comprises three major phases:
• Data preparation & augmentation
• Model learning (Supervised Fine-Tuning and KTO training)
• Inference & automated evaluation (including self-correction)
File structure to derive main components
• config/
• Deepspeed JSON configs for distributed training
• data/
• data/data_augment.py (prompt-based augmentation and classification)
• raw testsets (jsonl under data/testset) & example training sets
• prompts/
• Templates for problem formulation, augmentation, classification, self-correction
• sft/ and kto/
• sft/sft.py & script/run_sft.sh → multi-instruction supervised fine-tuning
• kto/kto.py & script/run_kto.sh → KTO (learning-to-define) training
• inference/
• inference.py, inference_q2f2c.py (various inference pipelines)
• merge_model.py (optional model ensemble/merging)
• utils/
• Helpers for augmentation, GPT-4 calls, solution extraction, templating, solver invocation
• script/
• Entrypoint shells for training workflows
High-level system components (to represent as distinct blocks)
A. Data Layer (blue)
1. Raw Datasets (data/testset/*.jsonl)
2. Example Training Data (data/trainset_example)
B. Data Processing & Augmentation (blue/green)
• data/data_augment.py → Generate definition prompts
• data/data_classification.py → Filter/classify problems
• prompts/*.py → Prompt templates for all modules
C. Model Learning (green)
1. Supervised Fine-Tuning (SFT)
– Module: sft/sft.py
– Entry: script/run_sft.sh
– Config: config/ds_config_zero3.json
2. Knowledge-to-Optimization (KTO) Training
– Module: kto/kto.py
– Entry: script/run_kto.sh
– Config: same Deepspeed config
D. Model Storage & Registry (purple)
• Hugging Face Hub (pre-trained base & LLMOPT fine-tuned models)
E. Inference & Self-Correction (orange/green)
• inference/inference.py (single-pass)
• inference/inference_q2f2c.py (question→formulation→code)
• inference/merge_model.py (ensemble or checkpoint merging)
F. Evaluation & Metrics (red)
• utils/solve.py → runs generated solver code to compute objective
• Reports Execution Rate (ER), Solving Accuracy (SA), Average Self-Corrections (AST)
• Raw Datasets → Data Processing & Augmentation → Formulated Prompt Examples
• Formulated Prompt Examples → SFT & KTO Training (via torchrun + Deepspeed) → Fine-tuned Models
• Fine-tuned Models ↔ Hugging Face Hub (push/pull)
• Fine-tuned Models + Prompt Templates + utils/gpt4_infer → Inference Pipeline
• Inference Pipeline → Generated Solver Code → utils/solve.py → Results
• Results + Self-Correction Loop (utils/gpt4_util + prompts/self_correction_prompt.py) → Improved Code
• Final Results → Evaluation Metrics Dashboard (TensorBoard logs)
Architectural patterns & design principles
• Modular pipeline: clearly separate Data → Training → Inference → Evaluation
• Reusable prompt templates: all prompts live in prompts/
• Config-driven training: Deepspeed JSON + shell scripts orchestrate distributed runs
• Self-correction feedback loop at inference time
• External-service integration: Hugging Face Model Hub, GPT-4 API