# prashantcsuthar-cmd/inferencedevs architecture

> Architecture diagram of the GitHub repository prashantcsuthar-cmd/inferencedevs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/prashantcsuthar-cmd/inferencedevs
- Repository: https://github.com/prashantcsuthar-cmd/inferencedevs

## Overview

ICE constrains local language-model generation by tracking a JSON grammar state, finding vocabulary tokens admissible in that state, masking other logits, and advancing the state after each generated token. The graph separates schema/grammar, token compatibility, decoding, and model loading. The README also describes diagnostics, validation, and a dashboard, but those paths are absent from the supplied tree, so they are not represented. The shown `main.py` excerpt is empty; runtime connections are limited to those visible in source.

Main entry points: [`src/decoding/engine.py`](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/decoding/engine.py) (Incremental decoder), [`src/tokenizer/compatibility.py`](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/compatibility.py) (Admissibility engine), [`src/schema/grammar.py`](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/grammar.py) (JSON transitions), [`src/schema/states.py`](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/states.py) (Grammar state)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_schema["Schema and grammar"]
  node_parser["Schema compiler<br/>[parser.py]"]
  node_grammar["JSON transitions<br/>[grammar.py]"]
  node_states["Grammar state<br/>[states.py]"]
end

subgraph group_tokens["Token compatibility"]
  node_compat["Admissibility engine<br/>[compatibility.py]"]
  node_trie["Token prefix trie<br/>[trie.py]"]
  node_vocab["Token vocabulary<br/>[vocabulary.py]"]
end

subgraph group_decoding["Constrained decoding"]
  node_engine["Incremental decoder<br/>[engine.py]"]
  node_processor["Logit masker"]
end

subgraph group_models["Model loading"]
  node_loader["Qwen loader<br/>[loader.py]"]
end

node_user(("Caller"))

node_user -.->|"requests generation"| node_engine
node_engine -->|"applies mask"| node_processor
node_engine -->|"gets valid tokens"| node_compat
node_engine -->|"advances state"| node_grammar
node_engine -->|"starts state"| node_states
node_engine -.->|"uses loaded model"| node_loader
node_processor -->|"gets valid tokens"| node_compat
node_compat -->|"tests transitions"| node_grammar
node_compat -->|"builds and traverses"| node_trie
node_compat -->|"builds and reads"| node_vocab
node_parser -->|"creates initial state"| node_states
node_grammar -->|"uses state types"| node_states
node_trie -->|"indexes vocabulary"| node_vocab

click node_engine "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/decoding/engine.py"
click node_processor "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/decoding/logits_processor.py"
click node_loader "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/models/loader.py"
click node_parser "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/parser.py"
click node_grammar "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/grammar.py"
click node_states "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/states.py"
click node_compat "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/compatibility.py"
click node_trie "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/trie.py"
click node_vocab "https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/vocabulary.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_parser,node_grammar,node_states toneBlue
class node_compat,node_trie,node_vocab toneAmber
class node_engine,node_processor toneMint
class node_loader toneRose
class node_user toneIndigo
```

## Components

### Schema and grammar

- [Schema compiler](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/parser.py): `src/schema/parser.py`
- [JSON transitions](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/grammar.py): `src/schema/grammar.py`
- [Grammar state](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/schema/states.py): `src/schema/states.py`

### Token compatibility

- [Admissibility engine](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/compatibility.py): `src/tokenizer/compatibility.py`
- [Token prefix trie](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/trie.py): `src/tokenizer/trie.py`
- [Token vocabulary](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/tokenizer/vocabulary.py): `src/tokenizer/vocabulary.py`

### Constrained decoding

- [Incremental decoder](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/decoding/engine.py): `src/decoding/engine.py`
- [Logit masker](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/decoding/logits_processor.py): `src/decoding/logits_processor.py`

### Model loading

- [Qwen loader](https://github.com/prashantcsuthar-cmd/inferencedevs/blob/main/src/models/loader.py): `src/models/loader.py`

### Other components

- **Caller**

## Connections

- Caller → Incremental decoder: requests generation
- Incremental decoder → Logit masker: applies mask (evidence: `src/decoding/engine.py`)
- Incremental decoder → Admissibility engine: gets valid tokens (evidence: `src/decoding/engine.py`)
- Incremental decoder → JSON transitions: advances state (evidence: `src/decoding/engine.py`)
- Incremental decoder → Grammar state: starts state (evidence: `src/decoding/engine.py`)
- Incremental decoder → Qwen loader: uses loaded model
- Logit masker → Admissibility engine: gets valid tokens (evidence: `src/decoding/logits_processor.py`)
- Admissibility engine → JSON transitions: tests transitions (evidence: `src/tokenizer/compatibility.py`)
- Admissibility engine → Token prefix trie: builds and traverses (evidence: `src/tokenizer/compatibility.py`)
- Admissibility engine → Token vocabulary: builds and reads (evidence: `src/tokenizer/compatibility.py`)
- Schema compiler → Grammar state: creates initial state (evidence: `src/schema/parser.py`)
- JSON transitions → Grammar state: uses state types (evidence: `src/schema/grammar.py`)
- Token prefix trie → Token vocabulary: indexes vocabulary (evidence: `src/tokenizer/trie.py`)
