How potamides/detikzify works
DeTi*k*Zify is a multimodal model and application for turning scientific figures or sketches into semantics-preserving TikZ programs. The main user workflow runs through the web UI, which invokes sampling or MCTS inference; generated code is compiled and rasterized, and MCTS can refine and rank candidate outputs using image similarity. The repository also includes model versions and adapters, dataset loaders, training, and evaluation metrics. Edges reflect README statements or the supplied source index and excerpts; unsampled implementation details are not inferred.
Main entry points: generate.py Generation pipeline · webui.py Web interface · modeling_detikzify.py Current multimodal model · modeling_adapter.py Cross-attention adapter
Components 21
User interface
- Web interface
detikzify/webui/webui.py - UI helpers
detikzify/webui/helpers.py - UI configuration
detikzify/webui/strings.py
Inference and rendering
- Generation pipeline
detikzify/infer/generate.py - TikZ compile and render
detikzify/infer/tikz.py - MCTS search
detikzify/mcts/montecarlo.py - Search nodes
detikzify/mcts/node.py - Shared runtime utilities
detikzify/util
Models
- Current model configuration
detikzify/model/configuration_detikzify.py - Current multimodal model
detikzify/model/modeling_detikzify.py - Current model processor
detikzify/model/processing_detikzify.py - Cross-attention adapter
detikzify/model/adapter/modeling_adapter.py - Adapter processor
detikzify/model/adapter/processing_adapter.py - Version 1 model
detikzify/model/v1/modeling_detikzify.py - Version 1 configuration
detikzify/model/v1/configuration_detikzify.py - Version 1 processor
detikzify/model/v1/processing_detikzify.py
Data and research
- Image similarity
detikzify/evaluate/imagesim.py - Other evaluation metrics
detikzify/evaluate - Figure datasets
detikzify/dataset - Model training
detikzify/train
Other components
- Figure or sketch user
Connections 16
- Figure or sketch user → Web interface: submits figure
- Web interface → Generation pipeline: invokes inference
detikzify/webui/webui.py - Web interface → UI helpers: uses helpers
detikzify/webui/webui.py - Web interface → UI configuration: uses settings
detikzify/webui/webui.py - Generation pipeline → TikZ compile and render: compiles output
detikzify/infer/generate.py - Generation pipeline → MCTS search: runs search
detikzify/infer/generate.py - Generation pipeline → Image similarity: scores candidates
detikzify/infer/generate.py - MCTS search → Search nodes: manages nodes
- Current multimodal model → Current model configuration: uses configuration
detikzify/model/modeling_detikzify.py - Current multimodal model → Cross-attention adapter: uses adapter
detikzify/model/modeling_detikzify.py - Image similarity → Cross-attention adapter: adapts vision model
detikzify/evaluate/imagesim.py - Version 1 model → Version 1 configuration: uses configuration
detikzify/model/v1/modeling_detikzify.py - Version 1 model → Version 1 processor: uses processor
detikzify/model/v1/modeling_detikzify.py - Cross-attention adapter → Adapter processor: pairs with processor
- Generation pipeline → Current multimodal model: generates with model
- TikZ compile and render → Shared runtime utilities: uses subprocess helpers
detikzify/infer/tikz.py