How joshuacnf/ctrl-g works
Ctrl-G performs constrained LLM generation: users define logical constraints, compile them to a DFA, and combine that automaton with an HMM-based logits processor during generation; generated candidates can then be extracted and ranked. The repository also supports HMM distillation from sampled text or vLLM text/vision-language outputs, including embedding-based clustering and HMM training. Sampled excerpts establish the core model and sampling behavior; some vLLM and training subsystem connections are based on the README and file layout, so uncertain internal wiring is omitted.
Main entry points: utils.py Constraint processor · tutorial_ctrlg.ipynb LLM generation · sample_data.py Text sampling · sample_data_instruct_vllm.py vLLM text sampling
Components 22
Constrained inference
- HMM model
ctrlg/hmm.py - Constraint processor
ctrlg/utils.py - LLM generation
tutorial_ctrlg.ipynb - Extract and rank
ctrlg/utils.py
Constraint modeling
- DFA builders
ctrlg/dfa.py - DFA operations
ctrlg/dfa.py - DFA model
ctrlg/dfa.py
Model sampling
- Text sampling
distillation/sample_data.py - vLLM text sampling
distillation_vllm/sample_data_instruct_vllm.py - VLM sampling
distillation_vllm/sample_data_instruct_vllm_img.py - Sampled sequences
HMM distillation
- Text embeddings
distillation_vllm/get_lvd_embedding.py - Image embeddings
distillation_vllm/get_lvd_embedding_img.py - LVD embeddings
- Embedding-based HMM
distillation/lvd_hmm.py - EM HMM training
distillation/train_hmm.py - vLLM EM training
distillation_vllm/train_hmm.py - Trained HMM
Other components
- User
- Base LLM
- vLLM server
- Ranked outputs
Connections 32
- User → DFA builders: specifies constraints
- DFA builders → DFA operations: builds automata
- DFA operations → DFA model: combines constraints
- User → Constraint processor: configures inference
- DFA model → Constraint processor: constrains states
- HMM model → Constraint processor: provides probabilities
- Constraint processor → LLM generation: adjusts logits
- User → LLM generation: starts generation
- Base LLM → LLM generation: generates candidates
- LLM generation → Extract and rank: returns sequences
- Extract and rank → Base LLM: scores candidates
- Extract and rank → Ranked outputs: ranks results
- User → Text sampling: provides prompts
- Base LLM → Text sampling: samples text
- User → vLLM text sampling: provides prompts
- vLLM server → vLLM text sampling: serves completions
- User → VLM sampling: provides messages
- vLLM server → VLM sampling: serves responses
- Text sampling → Sampled sequences: saves sequences
- vLLM text sampling → Sampled sequences: saves sequences
- VLM sampling → Sampled sequences: saves sequences
- Sampled sequences → EM HMM training: trains from sequences
- Sampled sequences → Text embeddings: provides token data
- Sampled sequences → Image embeddings: provides multimodal data
- Text embeddings → LVD embeddings: saves embeddings
- Image embeddings → LVD embeddings: saves embeddings
- Sampled sequences → Embedding-based HMM: provides sequences
- LVD embeddings → Embedding-based HMM: provides vectors
- Embedding-based HMM → Trained HMM: writes parameters
- EM HMM training → Trained HMM: updates parameters
- Sampled sequences → vLLM EM training: trains from sequences
- Trained HMM → HMM model: loads checkpoint