# joshuacnf/ctrl-g architecture

> Architecture diagram of the GitHub repository joshuacnf/ctrl-g, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/joshuacnf/ctrl-g
- Repository: https://github.com/joshuacnf/ctrl-g

## Overview

Ctrl-G performs constrained LLM generation: users define logical constraints, compile them to a DFA, and combine that automaton with an HMM-based logits processor during generation; generated candidates can then be extracted and ranked. The repository also supports HMM distillation from sampled text or vLLM text/vision-language outputs, including embedding-based clustering and HMM training. Sampled excerpts establish the core model and sampling behavior; some vLLM and training subsystem connections are based on the README and file layout, so uncertain internal wiring is omitted.

Main entry points: [`ctrlg/utils.py`](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/utils.py) (Constraint processor), [`tutorial_ctrlg.ipynb`](https://github.com/joshuacnf/ctrl-g/blob/main/tutorial_ctrlg.ipynb) (LLM generation), [`distillation/sample_data.py`](https://github.com/joshuacnf/ctrl-g/blob/main/distillation/sample_data.py) (Text sampling), [`distillation_vllm/sample_data_instruct_vllm.py`](https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/sample_data_instruct_vllm.py) (vLLM text sampling)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Constrained inference"]
  node_hmm_model["HMM model<br/>[hmm.py]"]
  node_logits_processor["Constraint processor<br/>[utils.py]"]
  node_generation["LLM generation"]
  node_output_processing["Extract and rank<br/>[utils.py]"]
end

subgraph group_constraints["Constraint modeling"]
  node_constraint_builders["DFA builders<br/>[dfa.py]"]
  node_dfa_ops["DFA operations<br/>[dfa.py]"]
  node_dfa_model["DFA model<br/>[dfa.py]"]
end

subgraph group_sampling["Model sampling"]
  node_text_sampler["Text sampling<br/>[sample_data.py]"]
  node_vllm_sampler["vLLM text sampling"]
  node_vlm_sampler["VLM sampling"]
  node_sample_data["Sampled sequences"]
end

subgraph group_distillation["HMM distillation"]
  node_embedding_text["Text embeddings"]
  node_embedding_vlm["Image embeddings"]
  node_embeddings["LVD embeddings"]
  node_lvd_training["Embedding-based HMM<br/>[lvd_hmm.py]"]
  node_em_training["EM HMM training<br/>[train_hmm.py]"]
  node_vllm_em_training["vLLM EM training<br/>[train_hmm.py]"]
  node_trained_hmm["Trained HMM"]
end

node_user(("User"))
node_base_llm["Base LLM"]
node_vllm["vLLM server"]
node_outputs["Ranked outputs"]

node_user -->|"specifies constraints"| node_constraint_builders
node_constraint_builders -->|"builds automata"| node_dfa_ops
node_dfa_ops -->|"combines constraints"| node_dfa_model
node_user -->|"configures inference"| node_logits_processor
node_dfa_model -->|"constrains states"| node_logits_processor
node_hmm_model -->|"provides probabilities"| node_logits_processor
node_logits_processor -->|"adjusts logits"| node_generation
node_user -->|"starts generation"| node_generation
node_base_llm -->|"generates candidates"| node_generation
node_generation -->|"returns sequences"| node_output_processing
node_output_processing -->|"scores candidates"| node_base_llm
node_output_processing -->|"ranks results"| node_outputs
node_user -->|"provides prompts"| node_text_sampler
node_base_llm -->|"samples text"| node_text_sampler
node_user -->|"provides prompts"| node_vllm_sampler
node_vllm -->|"serves completions"| node_vllm_sampler
node_user -->|"provides messages"| node_vlm_sampler
node_vllm -->|"serves responses"| node_vlm_sampler
node_text_sampler -->|"saves sequences"| node_sample_data
node_vllm_sampler -->|"saves sequences"| node_sample_data
node_vlm_sampler -->|"saves sequences"| node_sample_data
node_sample_data -->|"trains from sequences"| node_em_training
node_sample_data -->|"provides token data"| node_embedding_text
node_sample_data -.->|"provides multimodal data"| node_embedding_vlm
node_embedding_text -->|"saves embeddings"| node_embeddings
node_embedding_vlm -.->|"saves embeddings"| node_embeddings
node_sample_data -->|"provides sequences"| node_lvd_training
node_embeddings -->|"provides vectors"| node_lvd_training
node_lvd_training -->|"writes parameters"| node_trained_hmm
node_em_training -->|"updates parameters"| node_trained_hmm
node_sample_data -->|"trains from sequences"| node_vllm_em_training
node_trained_hmm -->|"loads checkpoint"| node_hmm_model

click node_constraint_builders "https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/dfa.py"
click node_dfa_ops "https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/dfa.py"
click node_dfa_model "https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/dfa.py"
click node_hmm_model "https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/hmm.py"
click node_logits_processor "https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/utils.py"
click node_generation "https://github.com/joshuacnf/ctrl-g/blob/main/tutorial_ctrlg.ipynb"
click node_output_processing "https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/utils.py"
click node_text_sampler "https://github.com/joshuacnf/ctrl-g/blob/main/distillation/sample_data.py"
click node_vllm_sampler "https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/sample_data_instruct_vllm.py"
click node_vlm_sampler "https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/sample_data_instruct_vllm_img.py"
click node_embedding_text "https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/get_lvd_embedding.py"
click node_embedding_vlm "https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/get_lvd_embedding_img.py"
click node_lvd_training "https://github.com/joshuacnf/ctrl-g/blob/main/distillation/lvd_hmm.py"
click node_em_training "https://github.com/joshuacnf/ctrl-g/blob/main/distillation/train_hmm.py"
click node_vllm_em_training "https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/train_hmm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_hmm_model,node_logits_processor,node_generation,node_output_processing,node_user toneBlue
class node_constraint_builders,node_dfa_ops,node_dfa_model toneAmber
class node_text_sampler,node_vllm_sampler,node_vlm_sampler,node_sample_data,node_vllm toneMint
class node_embedding_text,node_embedding_vlm,node_embeddings,node_lvd_training,node_em_training,node_vllm_em_training,node_trained_hmm toneRose
class node_base_llm,node_outputs toneIndigo
```

## Components

### Constrained inference

- [HMM model](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/hmm.py): `ctrlg/hmm.py`
- [Constraint processor](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/utils.py): `ctrlg/utils.py`
- [LLM generation](https://github.com/joshuacnf/ctrl-g/blob/main/tutorial_ctrlg.ipynb): `tutorial_ctrlg.ipynb`
- [Extract and rank](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/utils.py): `ctrlg/utils.py`

### Constraint modeling

- [DFA builders](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/dfa.py): `ctrlg/dfa.py`
- [DFA operations](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/dfa.py): `ctrlg/dfa.py`
- [DFA model](https://github.com/joshuacnf/ctrl-g/blob/main/ctrlg/dfa.py): `ctrlg/dfa.py`

### Model sampling

- [Text sampling](https://github.com/joshuacnf/ctrl-g/blob/main/distillation/sample_data.py): `distillation/sample_data.py`
- [vLLM text sampling](https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/sample_data_instruct_vllm.py): `distillation_vllm/sample_data_instruct_vllm.py`
- [VLM sampling](https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/sample_data_instruct_vllm_img.py): `distillation_vllm/sample_data_instruct_vllm_img.py`
- **Sampled sequences**

### HMM distillation

- [Text embeddings](https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/get_lvd_embedding.py): `distillation_vllm/get_lvd_embedding.py`
- [Image embeddings](https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/get_lvd_embedding_img.py): `distillation_vllm/get_lvd_embedding_img.py`
- **LVD embeddings**
- [Embedding-based HMM](https://github.com/joshuacnf/ctrl-g/blob/main/distillation/lvd_hmm.py): `distillation/lvd_hmm.py`
- [EM HMM training](https://github.com/joshuacnf/ctrl-g/blob/main/distillation/train_hmm.py): `distillation/train_hmm.py`
- [vLLM EM training](https://github.com/joshuacnf/ctrl-g/blob/main/distillation_vllm/train_hmm.py): `distillation_vllm/train_hmm.py`
- **Trained HMM**

### Other components

- **User**
- **Base LLM**
- **vLLM server**
- **Ranked outputs**

## Connections

- User → DFA builders: specifies constraints
- DFA builders → DFA operations: builds automata
- DFA operations → DFA model: combines constraints
- User → Constraint processor: configures inference
- DFA model → Constraint processor: constrains states
- HMM model → Constraint processor: provides probabilities
- Constraint processor → LLM generation: adjusts logits
- User → LLM generation: starts generation
- Base LLM → LLM generation: generates candidates
- LLM generation → Extract and rank: returns sequences
- Extract and rank → Base LLM: scores candidates
- Extract and rank → Ranked outputs: ranks results
- User → Text sampling: provides prompts
- Base LLM → Text sampling: samples text
- User → vLLM text sampling: provides prompts
- vLLM server → vLLM text sampling: serves completions
- User → VLM sampling: provides messages
- vLLM server → VLM sampling: serves responses
- Text sampling → Sampled sequences: saves sequences
- vLLM text sampling → Sampled sequences: saves sequences
- VLM sampling → Sampled sequences: saves sequences
- Sampled sequences → EM HMM training: trains from sequences
- Sampled sequences → Text embeddings: provides token data
- Sampled sequences → Image embeddings: provides multimodal data
- Text embeddings → LVD embeddings: saves embeddings
- Image embeddings → LVD embeddings: saves embeddings
- Sampled sequences → Embedding-based HMM: provides sequences
- LVD embeddings → Embedding-based HMM: provides vectors
- Embedding-based HMM → Trained HMM: writes parameters
- EM HMM training → Trained HMM: updates parameters
- Sampled sequences → vLLM EM training: trains from sequences
- Trained HMM → HMM model: loads checkpoint