How long-form-ai-video-generation/text-to-sketch works
This repository turns anime portraits into ordered vector-stroke sequences and trains or evaluates Sketchformer models on them. The main workflow runs from portrait acquisition through line-art extraction, filtering, vectorization/ordering, stroke-5 export, tokenization, and chunked training data; those data feed either the native PyTorch training path or the original Sketchformer handoff. It also supports model evaluation/export and preprocessing comparisons. The README documents several capabilities whose implementations were not sampled, so those branches are shown at subsystem level; edges are limited to documented workflow or sampled ownership.
Main entry points: workflow.py Preprocessing workflow · train.py Native train CLI · trainer.py Training and losses · evaluate.py Native evaluation
Components 27
Data preparation
- Download portraits
prep_data/download_data.py - Extract line art
prep_data/extract_sketches.py - Filter sketches
prep_data/filter_sketches.py - Preprocessing workflow
pipeline/workflow.py - Vectorize centerlines
pipeline/vectorization.py - Order drawing paths
pipeline/ordering.py - Stroke timing
pipeline/kinematics.py - Stroke-5 archives
- Build token dictionary
prep_data/sketch_token/create_token_dict.py - Token conversion
prep_data/prepare_sketchformer_tokens.py - Tok-dict chunks
Model training
- Native train CLI
scripts/sketchformer/train.py - Sequence data loaders
dataloaders/datamodule.py - Stroke sequence dataset
dataloaders/stroke_sequence_dataset.py - Model builder
builders/model_builder.py - PyTorch Sketchformer
models/sketchformer/model.py - Training and losses
core/trainer.py - Model checkpoints
core/checkpointing.py
Evaluation
- Native evaluation
scripts/sketchformer/evaluate.py - Reconstruction metrics
metrics/sketchformer/reconstruction.py - Model export
scripts/sketchformer/export.py - Preprocessing evaluation
metrics/preprocessing/evaluate_centerline_preprocessing.py
Legacy integration
- Legacy data preparation
prep_data/prepare_sketchformer.py - Original-model launcher
integrations/original_sketchformer/launcher.py
Other components
- Engineer
- Portrait source
- Original Sketchformer
Connections 37
- Engineer → Download portraits: starts
- Portrait source → Download portraits: provides images
- Download portraits → Extract line art: downloads portraits
- Extract line art → Filter sketches: writes sketches
- Engineer → Preprocessing workflow: runs
- Preprocessing workflow → Vectorize centerlines: invokes
- Preprocessing workflow → Order drawing paths: invokes
- Preprocessing workflow → Stroke timing: invokes
- Preprocessing workflow → Stroke-5 archives: writes
- Filter sketches → Preprocessing workflow: supplies sketches
- Stroke-5 archives → Build token dictionary: supplies strokes
- Engineer → Build token dictionary: creates
- Stroke-5 archives → Token conversion: converts
- Build token dictionary → Token conversion: encodes with
- Token conversion → Tok-dict chunks: writes chunks
- Engineer → Native train CLI: starts
- Native train CLI → Sequence data loaders: configures
- Sequence data loaders → Stroke sequence dataset: loads splits
- Tok-dict chunks → Stroke sequence dataset: provides sequences
- Native train CLI → Model builder: builds model
- Model builder → PyTorch Sketchformer: constructs
- Native train CLI → Training and losses: runs training
- Training and losses → Sequence data loaders: consumes batches
- Training and losses → PyTorch Sketchformer: optimizes
- Training and losses → Model checkpoints: saves
- Engineer → Native evaluation: starts
- Native evaluation → Sequence data loaders: loads evaluation data
- Native evaluation → PyTorch Sketchformer: runs inference
- Native evaluation → Reconstruction metrics: reports results
- Engineer → Model export: starts
- Model export → Model checkpoints: reads weights
- Engineer → Preprocessing evaluation: runs
- Preprocessing evaluation → Preprocessing workflow: evaluates preprocessing
- Engineer → Legacy data preparation: starts
- Legacy data preparation → Stroke-5 archives: reads strokes
- Engineer → Original-model launcher: launches
- Original-model launcher → Original Sketchformer: runs in Docker