How fvliang/dart works
DART is a speculative-decoding library and Gradio application for LLM inference. A user supplies a prompt and model weights; the application or Python API tokenizes it, then DART combines a draft-layer forward pass with n-gram-guided tree search before generating output. The model package also includes model-family implementations, configuration, templates, and KV-cache support. The README documents the main capabilities and API; some runtime internals were not sampled, so uncertain links are omitted.
Main entry points: dart_model.py DART Generation · tree_search.py Tree Search · app.py Gradio UI · main.py Python Demo
Components 18
User Entry
- Gradio UI
dart/app/app.py - Python Demo
main.py
DART Inference
- Prompt Templates
dart/model/template.py - DART Generation
dart/model/dart_model.py - Generation Utilities
dart/model/dart_utils.py
Tree Search
- Tree Search
dart/tree_search/tree_search.py - C++ Search
dart/tree_search/cpplib/cpp_search/searcher.cpp - N-gram Trie
dart/tree_search/cpplib/cpp_ngram/trie_ngram.cpp - N-gram Builder
dart/tree_search/ngram_build/ngram_build.py
Model Runtime
- Base Model
dart/model/base.py - DART Configurations
dart/model/dart_configs.py - KV Cache
dart/model/kv_cache.py - Llama Runtime
dart/model/modeling_llama_kv.py - Mixtral Runtime
dart/model/modeling_mixtral_kv.py - Qwen3 Runtime
dart/model/modeling_qwen3_kv.py - Qwen3 MoE Runtime
dart/model/modeling_qwen3_moe_kv.py - Llama3 DART Layer
dart/model/llama3_dart.py
Other components
- User
Connections 20
- User → Gradio UI: submits prompt
README.md - User → Python Demo: calls API
README.md - Gradio UI → Prompt Templates: formats prompt
dart/app/app.py - Gradio UI → DART Generation: streams generation
dart/app/app.py - Python Demo → DART Generation: loads and generates
main.py - Python Demo → Prompt Templates: formats prompt
main.py - DART Generation → Generation Utilities: uses helpers
dart/model/dart_model.py - Generation Utilities → Tree Search: invokes search
dart/model/dart_utils.py - Tree Search → C++ Search: searches candidates
dart/tree_search/tree_search.py - Tree Search → N-gram Trie: queries trie
dart/tree_search/tree_search.py - N-gram Builder → N-gram Trie: builds trie
- DART Generation → Base Model: wraps base model
dart/model/dart_model.py - DART Generation → DART Configurations: loads configuration
dart/model/dart_model.py - DART Generation → KV Cache: uses cache
dart/model/dart_model.py - DART Generation → Llama Runtime: selects runtime
dart/model/dart_model.py - DART Generation → Mixtral Runtime: selects runtime
dart/model/dart_model.py - DART Generation → Qwen3 Runtime: selects runtime
dart/model/dart_model.py - DART Generation → Qwen3 MoE Runtime: selects runtime
dart/model/dart_model.py - DART Generation → Llama3 DART Layer: uses draft layer
dart/model/dart_model.py - DART Generation → Tree Search: constructs draft tree