# long-form-ai-video-generation/text-to-sketch architecture

> Architecture diagram of the GitHub repository long-form-ai-video-generation/text-to-sketch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/long-form-ai-video-generation/text-to-sketch
- Repository: https://github.com/long-form-ai-video-generation/text-to-sketch

## Overview

This repository turns anime portraits into ordered vector-stroke sequences and trains or evaluates Sketchformer models on them. The main workflow runs from portrait acquisition through line-art extraction, filtering, vectorization/ordering, stroke-5 export, tokenization, and chunked training data; those data feed either the native PyTorch training path or the original Sketchformer handoff. It also supports model evaluation/export and preprocessing comparisons. The README documents several capabilities whose implementations were not sampled, so those branches are shown at subsystem level; edges are limited to documented workflow or sampled ownership.

Main entry points: [`pipeline/workflow.py`](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/workflow.py) (Preprocessing workflow), [`scripts/sketchformer/train.py`](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/train.py) (Native train CLI), [`core/trainer.py`](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/core/trainer.py) (Training and losses), [`scripts/sketchformer/evaluate.py`](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/evaluate.py) (Native evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_prep["Data preparation"]
  node_download["Download portraits<br/>[download_data.py]"]
  node_extract["Extract line art"]
  node_filter["Filter sketches<br/>[filter_sketches.py]"]
  node_workflow["Preprocessing workflow<br/>[workflow.py]"]
  node_vector["Vectorize centerlines<br/>[vectorization.py]"]
  node_ordering["Order drawing paths<br/>[ordering.py]"]
  node_kinematics["Stroke timing<br/>[kinematics.py]"]
  node_stroke5["Stroke-5 archives"]
  node_token_dict["Build token dictionary"]
  node_tokenize["Token conversion"]
  node_tokdata["Tok-dict chunks"]
end

subgraph group_model["Model training"]
  node_train_cli["Native train CLI<br/>[train.py]"]
  node_data_loader["Sequence data loaders<br/>[datamodule.py]"]
  node_dataset["Stroke sequence dataset"]
  node_model_builder["Model builder<br/>[model_builder.py]"]
  node_sketchformer["PyTorch Sketchformer<br/>[model.py]"]
  node_trainer["Training and losses<br/>[trainer.py]"]
  node_checkpoint["Model checkpoints<br/>[checkpointing.py]"]
end

subgraph group_eval["Evaluation"]
  node_evaluate["Native evaluation<br/>[evaluate.py]"]
  node_reconstruction["Reconstruction metrics<br/>[reconstruction.py]"]
  node_export["Model export<br/>[export.py]"]
  node_preprocess_eval["Preprocessing evaluation"]
end

subgraph group_legacy["Legacy integration"]
  node_legacy_prepare["Legacy data preparation"]
  node_legacy_launcher["Original-model launcher<br/>[launcher.py]"]
end

node_user(("Engineer"))
node_portraits[("Portrait source")]
node_original["Original Sketchformer"]

node_user -->|"starts"| node_download
node_portraits -->|"provides images"| node_download
node_download -->|"downloads portraits"| node_extract
node_extract -->|"writes sketches"| node_filter
node_user -->|"runs"| node_workflow
node_workflow -->|"invokes"| node_vector
node_workflow -->|"invokes"| node_ordering
node_workflow -->|"invokes"| node_kinematics
node_workflow -->|"writes"| node_stroke5
node_filter -->|"supplies sketches"| node_workflow
node_stroke5 -.->|"supplies strokes"| node_token_dict
node_user -->|"creates"| node_token_dict
node_stroke5 -->|"converts"| node_tokenize
node_token_dict -->|"encodes with"| node_tokenize
node_tokenize -->|"writes chunks"| node_tokdata
node_user -->|"starts"| node_train_cli
node_train_cli -->|"configures"| node_data_loader
node_data_loader -->|"loads splits"| node_dataset
node_tokdata -->|"provides sequences"| node_dataset
node_train_cli -->|"builds model"| node_model_builder
node_model_builder -->|"constructs"| node_sketchformer
node_train_cli -->|"runs training"| node_trainer
node_trainer -->|"consumes batches"| node_data_loader
node_trainer -->|"optimizes"| node_sketchformer
node_trainer -->|"saves"| node_checkpoint
node_user -->|"starts"| node_evaluate
node_evaluate -->|"loads evaluation data"| node_data_loader
node_evaluate -->|"runs inference"| node_sketchformer
node_evaluate -->|"reports results"| node_reconstruction
node_user -->|"starts"| node_export
node_export -->|"reads weights"| node_checkpoint
node_user -->|"runs"| node_preprocess_eval
node_preprocess_eval -.->|"evaluates preprocessing"| node_workflow
node_user -->|"starts"| node_legacy_prepare
node_legacy_prepare -->|"reads strokes"| node_stroke5
node_user -->|"launches"| node_legacy_launcher
node_legacy_launcher -->|"runs in Docker"| node_original

click node_download "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/download_data.py"
click node_extract "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/extract_sketches.py"
click node_filter "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/filter_sketches.py"
click node_workflow "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/workflow.py"
click node_vector "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/vectorization.py"
click node_ordering "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/ordering.py"
click node_kinematics "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/kinematics.py"
click node_token_dict "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/sketch_token/create_token_dict.py"
click node_tokenize "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/prepare_sketchformer_tokens.py"
click node_train_cli "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/train.py"
click node_data_loader "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/dataloaders/datamodule.py"
click node_dataset "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/dataloaders/stroke_sequence_dataset.py"
click node_model_builder "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/builders/model_builder.py"
click node_sketchformer "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/models/sketchformer/model.py"
click node_trainer "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/core/trainer.py"
click node_checkpoint "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/core/checkpointing.py"
click node_evaluate "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/evaluate.py"
click node_reconstruction "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/metrics/sketchformer/reconstruction.py"
click node_export "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/export.py"
click node_preprocess_eval "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/metrics/preprocessing/evaluate_centerline_preprocessing.py"
click node_legacy_prepare "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/prepare_sketchformer.py"
click node_legacy_launcher "https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/integrations/original_sketchformer/launcher.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_download,node_extract,node_filter,node_workflow,node_vector,node_ordering,node_kinematics,node_stroke5,node_token_dict,node_tokenize,node_tokdata toneBlue
class node_train_cli,node_data_loader,node_dataset,node_model_builder,node_sketchformer,node_trainer,node_checkpoint,node_portraits toneAmber
class node_evaluate,node_reconstruction,node_export,node_preprocess_eval toneMint
class node_legacy_prepare,node_legacy_launcher toneRose
class node_user,node_original toneIndigo
```

## Components

### Data preparation

- [Download portraits](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/download_data.py): `prep_data/download_data.py`
- [Extract line art](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/extract_sketches.py): `prep_data/extract_sketches.py`
- [Filter sketches](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/filter_sketches.py): `prep_data/filter_sketches.py`
- [Preprocessing workflow](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/workflow.py): `pipeline/workflow.py`
- [Vectorize centerlines](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/vectorization.py): `pipeline/vectorization.py`
- [Order drawing paths](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/ordering.py): `pipeline/ordering.py`
- [Stroke timing](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/pipeline/kinematics.py): `pipeline/kinematics.py`
- **Stroke-5 archives**
- [Build token dictionary](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/sketch_token/create_token_dict.py): `prep_data/sketch_token/create_token_dict.py`
- [Token conversion](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/prepare_sketchformer_tokens.py): `prep_data/prepare_sketchformer_tokens.py`
- **Tok-dict chunks**

### Model training

- [Native train CLI](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/train.py): `scripts/sketchformer/train.py`
- [Sequence data loaders](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/dataloaders/datamodule.py): `dataloaders/datamodule.py`
- [Stroke sequence dataset](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/dataloaders/stroke_sequence_dataset.py): `dataloaders/stroke_sequence_dataset.py`
- [Model builder](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/builders/model_builder.py): `builders/model_builder.py`
- [PyTorch Sketchformer](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/models/sketchformer/model.py): `models/sketchformer/model.py`
- [Training and losses](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/core/trainer.py): `core/trainer.py`
- [Model checkpoints](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/core/checkpointing.py): `core/checkpointing.py`

### Evaluation

- [Native evaluation](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/evaluate.py): `scripts/sketchformer/evaluate.py`
- [Reconstruction metrics](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/metrics/sketchformer/reconstruction.py): `metrics/sketchformer/reconstruction.py`
- [Model export](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/scripts/sketchformer/export.py): `scripts/sketchformer/export.py`
- [Preprocessing evaluation](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/metrics/preprocessing/evaluate_centerline_preprocessing.py): `metrics/preprocessing/evaluate_centerline_preprocessing.py`

### Legacy integration

- [Legacy data preparation](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/prep_data/prepare_sketchformer.py): `prep_data/prepare_sketchformer.py`
- [Original-model launcher](https://github.com/long-form-ai-video-generation/text-to-sketch/blob/main/integrations/original_sketchformer/launcher.py): `integrations/original_sketchformer/launcher.py`

### Other components

- **Engineer**
- **Portrait source**
- **Original Sketchformer**

## Connections

- Engineer → Download portraits: starts
- Portrait source → Download portraits: provides images
- Download portraits → Extract line art: downloads portraits
- Extract line art → Filter sketches: writes sketches
- Engineer → Preprocessing workflow: runs
- Preprocessing workflow → Vectorize centerlines: invokes
- Preprocessing workflow → Order drawing paths: invokes
- Preprocessing workflow → Stroke timing: invokes
- Preprocessing workflow → Stroke-5 archives: writes
- Filter sketches → Preprocessing workflow: supplies sketches
- Stroke-5 archives → Build token dictionary: supplies strokes
- Engineer → Build token dictionary: creates
- Stroke-5 archives → Token conversion: converts
- Build token dictionary → Token conversion: encodes with
- Token conversion → Tok-dict chunks: writes chunks
- Engineer → Native train CLI: starts
- Native train CLI → Sequence data loaders: configures
- Sequence data loaders → Stroke sequence dataset: loads splits
- Tok-dict chunks → Stroke sequence dataset: provides sequences
- Native train CLI → Model builder: builds model
- Model builder → PyTorch Sketchformer: constructs
- Native train CLI → Training and losses: runs training
- Training and losses → Sequence data loaders: consumes batches
- Training and losses → PyTorch Sketchformer: optimizes
- Training and losses → Model checkpoints: saves
- Engineer → Native evaluation: starts
- Native evaluation → Sequence data loaders: loads evaluation data
- Native evaluation → PyTorch Sketchformer: runs inference
- Native evaluation → Reconstruction metrics: reports results
- Engineer → Model export: starts
- Model export → Model checkpoints: reads weights
- Engineer → Preprocessing evaluation: runs
- Preprocessing evaluation → Preprocessing workflow: evaluates preprocessing
- Engineer → Legacy data preparation: starts
- Legacy data preparation → Stroke-5 archives: reads strokes
- Engineer → Original-model launcher: launches
- Original-model launcher → Original Sketchformer: runs in Docker