# guandeh17/self-forcing architecture

> Architecture diagram of the GitHub repository guandeh17/self-forcing, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/guandeh17/self-forcing
- Repository: https://github.com/guandeh17/self-forcing

## Overview

Self Forcing is a research system for autoregressive video diffusion, with GUI and CLI generation as its user-facing path and training methods as a second major capability. Inference encodes prompts, performs causal or bidirectional diffusion sampling, then decodes video; causal sampling maintains KV caches to support chunked generation. The repository also includes Wan text-to-video and image-to-video runtimes, training objectives, and data preparation utilities. Some training components are identified from the tree and README only; their internal calls are not asserted.

Main entry points: [`utils/wan_wrapper.py`](https://github.com/guandeh17/self-forcing/blob/main/utils/wan_wrapper.py) (Wan wrappers), [`trainer`](https://github.com/guandeh17/self-forcing/tree/main/trainer) (Training loops), [`wan/modules/model.py`](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/model.py) (Wan transformer), [`pipeline/causal_diffusion_inference.py`](https://github.com/guandeh17/self-forcing/blob/main/pipeline/causal_diffusion_inference.py) (Causal sampling)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User entry points"]
  node_gui["GUI demo<br/>[demo.py]"]
  node_cli["CLI inference<br/>[inference.py]"]
  node_traincmd["Training command<br/>[train.py]"]
end

subgraph group_sampling["Generation pipelines"]
  node_causal["Causal sampling"]
  node_causal_plain["Causal inference"]
  node_bidir["Bidirectional sampling"]
  node_bidir_plain["Bidirectional inference"]
  node_trainpipe["Self-forcing rollout"]
  node_solver["Flow solvers<br/>[fm_solvers.py]"]
  node_unipc["UniPC solver"]
end

subgraph group_wan["Wan model runtime"]
  node_wrapper["Wan wrappers<br/>[wan_wrapper.py]"]
  node_text["Text encoder<br/>[t5.py]"]
  node_token["Tokenizers<br/>[tokenizers.py]"]
  node_dit["Wan transformer<br/>[model.py]"]
  node_causal_dit["Causal transformer<br/>[causal_model.py]"]
  node_attention["Attention kernels<br/>[attention.py]"]
  node_vae["Video VAE<br/>[vae.py]"]
  node_wan_t2v["Wan text-to-video<br/>[text2video.py]"]
  node_wan_i2v["Wan image-to-video<br/>[image2video.py]"]
  node_parallel["Context parallelism"]
end

subgraph group_learning["Training methods"]
  node_dmd["DMD objective<br/>[dmd.py]"]
  node_causvid["CausVid objective<br/>[causvid.py]"]
  node_gan["GAN objective<br/>[gan.py]"]
  node_sid["SID objective<br/>[sid.py]"]
  node_ode["ODE regression<br/>[ode_regression.py]"]
  node_trainer["Training loops"]
end

subgraph group_data["Training data"]
  node_dataset["Prompt dataset<br/>[dataset.py]"]
  node_lmdb[("LMDB access<br/>[lmdb.py]")]
end

node_user(("User"))
node_actor(("External checkpoints"))

node_user -->|"uses"| node_gui
node_user -->|"runs"| node_cli
node_gui -->|"loads models"| node_wrapper
node_gui -.->|"uses pipeline"| node_causal
node_cli -.->|"selects pipeline"| node_causal
node_cli -->|"reads prompts"| node_dataset
node_causal -->|"calls"| node_wrapper
node_causal -->|"schedules steps"| node_solver
node_causal -->|"uses solver"| node_unipc
node_causal_plain -->|"calls"| node_wrapper
node_bidir -->|"calls"| node_wrapper
node_bidir -->|"schedules steps"| node_solver
node_bidir -->|"uses solver"| node_unipc
node_bidir_plain -->|"uses models"| node_wrapper
node_wrapper -->|"loads encoder"| node_text
node_wrapper -->|"uses tokenizer"| node_token
node_wrapper -->|"loads model"| node_dit
node_wrapper -->|"loads causal model"| node_causal_dit
node_wrapper -->|"loads decoder"| node_vae
node_causal_dit -->|"reuses layers"| node_dit
node_causal_dit -->|"uses attention"| node_attention
node_dit -->|"uses attention"| node_attention
node_wan_t2v -->|"runs model"| node_dit
node_wan_i2v -->|"runs model"| node_dit
node_wan_i2v -->|"enables parallelism"| node_parallel
node_parallel -->|"patches forward"| node_dit
node_traincmd -.->|"starts training"| node_trainer
node_trainer -.->|"uses rollout"| node_trainpipe
node_trainer -->|"loads prompts"| node_dataset
node_dataset -->|"reads records"| node_lmdb
node_trainer -.->|"trains objective"| node_dmd
node_trainer -.->|"trains objective"| node_causvid
node_trainer -.->|"trains objective"| node_gan
node_trainer -.->|"trains objective"| node_sid
node_trainer -.->|"trains objective"| node_ode
node_actor -.->|"provides weights"| node_wrapper

click node_gui "https://github.com/guandeh17/self-forcing/blob/main/demo.py"
click node_cli "https://github.com/guandeh17/self-forcing/blob/main/inference.py"
click node_traincmd "https://github.com/guandeh17/self-forcing/blob/main/train.py"
click node_causal "https://github.com/guandeh17/self-forcing/blob/main/pipeline/causal_diffusion_inference.py"
click node_causal_plain "https://github.com/guandeh17/self-forcing/blob/main/pipeline/causal_inference.py"
click node_bidir "https://github.com/guandeh17/self-forcing/blob/main/pipeline/bidirectional_diffusion_inference.py"
click node_bidir_plain "https://github.com/guandeh17/self-forcing/blob/main/pipeline/bidirectional_inference.py"
click node_trainpipe "https://github.com/guandeh17/self-forcing/blob/main/pipeline/self_forcing_training.py"
click node_solver "https://github.com/guandeh17/self-forcing/blob/main/wan/utils/fm_solvers.py"
click node_unipc "https://github.com/guandeh17/self-forcing/blob/main/wan/utils/fm_solvers_unipc.py"
click node_wrapper "https://github.com/guandeh17/self-forcing/blob/main/utils/wan_wrapper.py"
click node_text "https://github.com/guandeh17/self-forcing/blob/main/wan/modules/t5.py"
click node_token "https://github.com/guandeh17/self-forcing/blob/main/wan/modules/tokenizers.py"
click node_dit "https://github.com/guandeh17/self-forcing/blob/main/wan/modules/model.py"
click node_causal_dit "https://github.com/guandeh17/self-forcing/blob/main/wan/modules/causal_model.py"
click node_attention "https://github.com/guandeh17/self-forcing/blob/main/wan/modules/attention.py"
click node_vae "https://github.com/guandeh17/self-forcing/blob/main/wan/modules/vae.py"
click node_wan_t2v "https://github.com/guandeh17/self-forcing/blob/main/wan/text2video.py"
click node_wan_i2v "https://github.com/guandeh17/self-forcing/blob/main/wan/image2video.py"
click node_parallel "https://github.com/guandeh17/self-forcing/blob/main/wan/distributed/xdit_context_parallel.py"
click node_dmd "https://github.com/guandeh17/self-forcing/blob/main/model/dmd.py"
click node_causvid "https://github.com/guandeh17/self-forcing/blob/main/model/causvid.py"
click node_gan "https://github.com/guandeh17/self-forcing/blob/main/model/gan.py"
click node_sid "https://github.com/guandeh17/self-forcing/blob/main/model/sid.py"
click node_ode "https://github.com/guandeh17/self-forcing/blob/main/model/ode_regression.py"
click node_trainer "https://github.com/guandeh17/self-forcing/tree/main/trainer"
click node_dataset "https://github.com/guandeh17/self-forcing/blob/main/utils/dataset.py"
click node_lmdb "https://github.com/guandeh17/self-forcing/blob/main/utils/lmdb.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gui,node_cli,node_traincmd,node_user toneBlue
class node_causal,node_causal_plain,node_bidir,node_bidir_plain,node_trainpipe,node_solver,node_unipc toneAmber
class node_wrapper,node_text,node_token,node_dit,node_causal_dit,node_attention,node_vae,node_wan_t2v,node_wan_i2v,node_parallel toneMint
class node_dmd,node_causvid,node_gan,node_sid,node_ode,node_trainer toneRose
class node_dataset,node_lmdb,node_actor toneIndigo
```

## Components

### User entry points

- [GUI demo](https://github.com/guandeh17/self-forcing/blob/main/demo.py): `demo.py`
- [CLI inference](https://github.com/guandeh17/self-forcing/blob/main/inference.py): `inference.py`
- [Training command](https://github.com/guandeh17/self-forcing/blob/main/train.py): `train.py`

### Generation pipelines

- [Causal sampling](https://github.com/guandeh17/self-forcing/blob/main/pipeline/causal_diffusion_inference.py): `pipeline/causal_diffusion_inference.py`
- [Causal inference](https://github.com/guandeh17/self-forcing/blob/main/pipeline/causal_inference.py): `pipeline/causal_inference.py`
- [Bidirectional sampling](https://github.com/guandeh17/self-forcing/blob/main/pipeline/bidirectional_diffusion_inference.py): `pipeline/bidirectional_diffusion_inference.py`
- [Bidirectional inference](https://github.com/guandeh17/self-forcing/blob/main/pipeline/bidirectional_inference.py): `pipeline/bidirectional_inference.py`
- [Self-forcing rollout](https://github.com/guandeh17/self-forcing/blob/main/pipeline/self_forcing_training.py): `pipeline/self_forcing_training.py`
- [Flow solvers](https://github.com/guandeh17/self-forcing/blob/main/wan/utils/fm_solvers.py): `wan/utils/fm_solvers.py`
- [UniPC solver](https://github.com/guandeh17/self-forcing/blob/main/wan/utils/fm_solvers_unipc.py): `wan/utils/fm_solvers_unipc.py`

### Wan model runtime

- [Wan wrappers](https://github.com/guandeh17/self-forcing/blob/main/utils/wan_wrapper.py): `utils/wan_wrapper.py`
- [Text encoder](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/t5.py): `wan/modules/t5.py`
- [Tokenizers](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/tokenizers.py): `wan/modules/tokenizers.py`
- [Wan transformer](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/model.py): `wan/modules/model.py`
- [Causal transformer](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/causal_model.py): `wan/modules/causal_model.py`
- [Attention kernels](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/attention.py): `wan/modules/attention.py`
- [Video VAE](https://github.com/guandeh17/self-forcing/blob/main/wan/modules/vae.py): `wan/modules/vae.py`
- [Wan text-to-video](https://github.com/guandeh17/self-forcing/blob/main/wan/text2video.py): `wan/text2video.py`
- [Wan image-to-video](https://github.com/guandeh17/self-forcing/blob/main/wan/image2video.py): `wan/image2video.py`
- [Context parallelism](https://github.com/guandeh17/self-forcing/blob/main/wan/distributed/xdit_context_parallel.py): `wan/distributed/xdit_context_parallel.py`

### Training methods

- [DMD objective](https://github.com/guandeh17/self-forcing/blob/main/model/dmd.py): `model/dmd.py`
- [CausVid objective](https://github.com/guandeh17/self-forcing/blob/main/model/causvid.py): `model/causvid.py`
- [GAN objective](https://github.com/guandeh17/self-forcing/blob/main/model/gan.py): `model/gan.py`
- [SID objective](https://github.com/guandeh17/self-forcing/blob/main/model/sid.py): `model/sid.py`
- [ODE regression](https://github.com/guandeh17/self-forcing/blob/main/model/ode_regression.py): `model/ode_regression.py`
- [Training loops](https://github.com/guandeh17/self-forcing/tree/main/trainer): `trainer`

### Training data

- [Prompt dataset](https://github.com/guandeh17/self-forcing/blob/main/utils/dataset.py): `utils/dataset.py`
- [LMDB access](https://github.com/guandeh17/self-forcing/blob/main/utils/lmdb.py): `utils/lmdb.py`

### Other components

- **User**
- **External checkpoints**

## Connections

- User → GUI demo: uses (evidence: `README.md`)
- User → CLI inference: runs (evidence: `README.md`)
- GUI demo → Wan wrappers: loads models (evidence: `demo.py`)
- GUI demo → Causal sampling: uses pipeline
- CLI inference → Causal sampling: selects pipeline
- CLI inference → Prompt dataset: reads prompts (evidence: `inference.py`)
- Causal sampling → Wan wrappers: calls (evidence: `pipeline/causal_diffusion_inference.py`)
- Causal sampling → Flow solvers: schedules steps (evidence: `pipeline/causal_diffusion_inference.py`)
- Causal sampling → UniPC solver: uses solver (evidence: `pipeline/causal_diffusion_inference.py`)
- Causal inference → Wan wrappers: calls (evidence: `pipeline/causal_inference.py`)
- Bidirectional sampling → Wan wrappers: calls (evidence: `pipeline/bidirectional_diffusion_inference.py`)
- Bidirectional sampling → Flow solvers: schedules steps (evidence: `pipeline/bidirectional_diffusion_inference.py`)
- Bidirectional sampling → UniPC solver: uses solver (evidence: `pipeline/bidirectional_diffusion_inference.py`)
- Bidirectional inference → Wan wrappers: uses models (evidence: `pipeline/bidirectional_inference.py`)
- Wan wrappers → Text encoder: loads encoder (evidence: `utils/wan_wrapper.py`)
- Wan wrappers → Tokenizers: uses tokenizer (evidence: `utils/wan_wrapper.py`)
- Wan wrappers → Wan transformer: loads model (evidence: `utils/wan_wrapper.py`)
- Wan wrappers → Causal transformer: loads causal model (evidence: `utils/wan_wrapper.py`)
- Wan wrappers → Video VAE: loads decoder (evidence: `utils/wan_wrapper.py`)
- Causal transformer → Wan transformer: reuses layers (evidence: `wan/modules/causal_model.py`)
- Causal transformer → Attention kernels: uses attention (evidence: `wan/modules/causal_model.py`)
- Wan transformer → Attention kernels: uses attention (evidence: `wan/modules/model.py`)
- Wan text-to-video → Wan transformer: runs model (evidence: `wan/text2video.py`)
- Wan image-to-video → Wan transformer: runs model (evidence: `wan/image2video.py`)
- Wan image-to-video → Context parallelism: enables parallelism (evidence: `wan/image2video.py`)
- Context parallelism → Wan transformer: patches forward (evidence: `wan/distributed/xdit_context_parallel.py`)
- Training command → Training loops: starts training
- Training loops → Self-forcing rollout: uses rollout
- Training loops → Prompt dataset: loads prompts (evidence: `trainer/distillation.py`)
- Prompt dataset → LMDB access: reads records (evidence: `utils/dataset.py`)
- Training loops → DMD objective: trains objective
- Training loops → CausVid objective: trains objective
- Training loops → GAN objective: trains objective
- Training loops → SID objective: trains objective
- Training loops → ODE regression: trains objective
- External checkpoints → Wan wrappers: provides weights
