# picsart-ai-research/streamingt2v architecture

> Architecture diagram of the GitHub repository picsart-ai-research/streamingt2v, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/picsart-ai-research/streamingt2v
- Repository: https://github.com/picsart-ai-research/streamingt2v

## Overview

StreamingSVD turns input images into long, temporally consistent videos through autoregressive image-to-video generation, followed by video enhancement (including optional randomized blending) and frame interpolation. The inference entry point prepares the input and components; the StreamingSVD trainer owns generation, while enhancement and interpolation are distinct downstream capabilities. The graph also shows the model-loading and diffusion-model layers. Some implementation excerpts are partial, so uncertain runtime links are marked dashed with no evidence path.

Main entry points: [`code/inference_i2v.py`](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/inference_i2v.py) (Inference pipeline), [`code/diffusion_trainer/streaming_svd.py`](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/diffusion_trainer/streaming_svd.py) (StreamingSVD trainer), [`code/i2v_enhance/i2v_enhance_interface.py`](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/i2v_enhance_interface.py) (Enhancement interface), [`code/i2v_enhance/unet_3d_blocks.py`](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/unet_3d_blocks.py) (3D UNet blocks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Inference"]
  node_entry["Inference pipeline<br/>[inference_i2v.py]"]
  node_image_data["Image dataset"]
  node_data_module["Video data module"]
end

subgraph group_generation["Video generation"]
  node_loader["Module loader<br/>[module_loader.py]"]
  node_trainer["StreamingSVD trainer<br/>[streaming_svd.py]"]
end

subgraph group_model["Model internals"]
  node_diffusion["Video diffusion model<br/>[video_model.py]"]
  node_unet["Diffusion UNet<br/>[openaimodel.py]"]
  node_video_attention["Video attention<br/>[video_attention.py]"]
  node_attention["Attention layers<br/>[attention.py]"]
  node_autoencoder["Video autoencoder<br/>[temporal_ae.py]"]
end

subgraph group_enhance["Enhancement"]
  node_enhance_interface["Enhancement interface"]
  node_enhance_pipeline["I2VGen-XL pipeline"]
  node_enhance_unet["Enhancement UNet<br/>[unet_i2vgen_xl.py]"]
  node_enhance_blocks["3D UNet blocks<br/>[unet_3d_blocks.py]"]
  node_enhance_transformer["Spatial transformer<br/>[transformer_2d.py]"]
  node_temporal_transformer["Temporal transformer"]
  node_vfi["Frame interpolation<br/>[Trainer.py]"]
end

subgraph group_output["Output"]
  node_results["Result processing"]
end

node_user(("Image input"))
node_video["Generated video"]

node_user -->|"provides image"| node_entry
node_entry -->|"creates dataset"| node_image_data
node_entry -->|"configures input"| node_data_module
node_data_module -->|"uses factory"| node_image_data
node_entry -->|"runs prediction"| node_trainer
node_trainer -->|"uses loader"| node_loader
node_trainer -.->|"runs diffusion"| node_diffusion
node_diffusion -->|"uses backbone"| node_unet
node_unet -->|"uses attention"| node_video_attention
node_video_attention -->|"uses layers"| node_attention
node_trainer -->|"decodes frames"| node_autoencoder
node_entry -->|"initializes enhancement"| node_enhance_interface
node_enhance_interface -->|"initializes pipeline"| node_enhance_pipeline
node_enhance_pipeline -->|"runs enhancement"| node_enhance_unet
node_enhance_unet -->|"uses blocks"| node_enhance_blocks
node_enhance_blocks -->|"uses transformer"| node_enhance_transformer
node_enhance_blocks -->|"uses temporal layers"| node_temporal_transformer
node_enhance_transformer -->|"uses temporal layers"| node_temporal_transformer
node_entry -->|"initializes interpolation"| node_vfi
node_trainer -->|"enhances generated video"| node_enhance_interface
node_enhance_interface -->|"interpolates frames"| node_vfi
node_entry -.->|"processes results"| node_results
node_results -.->|"saves video"| node_video

click node_entry "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/inference_i2v.py"
click node_image_data "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/dataloader/single_image_dataset.py"
click node_data_module "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/dataloader/video_data_module.py"
click node_loader "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/modules/loader/module_loader.py"
click node_trainer "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/diffusion_trainer/streaming_svd.py"
click node_diffusion "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/diffusion/video_model.py"
click node_unet "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/diffusionmodules/openaimodel.py"
click node_video_attention "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/video_attention.py"
click node_attention "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/attention.py"
click node_autoencoder "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/autoencoding/temporal_ae.py"
click node_enhance_interface "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/i2v_enhance_interface.py"
click node_enhance_pipeline "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/pipeline_i2vgen_xl.py"
click node_enhance_unet "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/unet_i2vgen_xl.py"
click node_enhance_blocks "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/unet_3d_blocks.py"
click node_enhance_transformer "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/transformer_2d.py"
click node_temporal_transformer "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/transformer_temporal.py"
click node_vfi "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/thirdparty/VFI/Trainer.py"
click node_results "https://github.com/picsart-ai-research/streamingt2v/blob/main/code/utils/result_processor.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_image_data,node_data_module toneBlue
class node_loader,node_trainer toneAmber
class node_diffusion,node_unet,node_video_attention,node_attention,node_autoencoder toneMint
class node_enhance_interface,node_enhance_pipeline,node_enhance_unet,node_enhance_blocks,node_enhance_transformer,node_temporal_transformer,node_vfi toneRose
class node_results,node_user,node_video toneIndigo
```

## Components

### Inference

- [Inference pipeline](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/inference_i2v.py): `code/inference_i2v.py`
- [Image dataset](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/dataloader/single_image_dataset.py): `code/dataloader/single_image_dataset.py`
- [Video data module](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/dataloader/video_data_module.py): `code/dataloader/video_data_module.py`

### Video generation

- [Module loader](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/modules/loader/module_loader.py): `code/modules/loader/module_loader.py`
- [StreamingSVD trainer](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/diffusion_trainer/streaming_svd.py): `code/diffusion_trainer/streaming_svd.py`

### Model internals

- [Video diffusion model](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/diffusion/video_model.py): `code/models/diffusion/video_model.py`
- [Diffusion UNet](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/diffusionmodules/openaimodel.py): `code/models/svd/sgm/modules/diffusionmodules/openaimodel.py`
- [Video attention](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/video_attention.py): `code/models/svd/sgm/modules/video_attention.py`
- [Attention layers](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/attention.py): `code/models/svd/sgm/modules/attention.py`
- [Video autoencoder](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/models/svd/sgm/modules/autoencoding/temporal_ae.py): `code/models/svd/sgm/modules/autoencoding/temporal_ae.py`

### Enhancement

- [Enhancement interface](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/i2v_enhance_interface.py): `code/i2v_enhance/i2v_enhance_interface.py`
- [I2VGen-XL pipeline](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/pipeline_i2vgen_xl.py): `code/i2v_enhance/pipeline_i2vgen_xl.py`
- [Enhancement UNet](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/unet_i2vgen_xl.py): `code/i2v_enhance/unet_i2vgen_xl.py`
- [3D UNet blocks](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/unet_3d_blocks.py): `code/i2v_enhance/unet_3d_blocks.py`
- [Spatial transformer](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/transformer_2d.py): `code/i2v_enhance/transformer_2d.py`
- [Temporal transformer](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/transformer_temporal.py): `code/i2v_enhance/transformer_temporal.py`
- [Frame interpolation](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/i2v_enhance/thirdparty/VFI/Trainer.py): `code/i2v_enhance/thirdparty/VFI/Trainer.py`

### Output

- [Result processing](https://github.com/picsart-ai-research/streamingt2v/blob/main/code/utils/result_processor.py): `code/utils/result_processor.py`

### Other components

- **Image input**
- **Generated video**

## Connections

- Image input → Inference pipeline: provides image (evidence: `README.md`)
- Inference pipeline → Image dataset: creates dataset (evidence: `code/inference_i2v.py`)
- Inference pipeline → Video data module: configures input (evidence: `code/inference_i2v.py`)
- Video data module → Image dataset: uses factory (evidence: `code/dataloader/video_data_module.py`)
- Inference pipeline → StreamingSVD trainer: runs prediction (evidence: `code/inference_i2v.py`)
- StreamingSVD trainer → Module loader: uses loader (evidence: `code/diffusion_trainer/streaming_svd.py`)
- StreamingSVD trainer → Video diffusion model: runs diffusion
- Video diffusion model → Diffusion UNet: uses backbone (evidence: `code/models/diffusion/video_model.py`)
- Diffusion UNet → Video attention: uses attention (evidence: `code/models/svd/sgm/modules/diffusionmodules/openaimodel.py`)
- Video attention → Attention layers: uses layers (evidence: `code/models/svd/sgm/modules/video_attention.py`)
- StreamingSVD trainer → Video autoencoder: decodes frames (evidence: `code/diffusion_trainer/streaming_svd.py`)
- Inference pipeline → Enhancement interface: initializes enhancement (evidence: `code/inference_i2v.py`)
- Enhancement interface → I2VGen-XL pipeline: initializes pipeline (evidence: `code/i2v_enhance/i2v_enhance_interface.py`)
- I2VGen-XL pipeline → Enhancement UNet: runs enhancement (evidence: `code/i2v_enhance/pipeline_i2vgen_xl.py`)
- Enhancement UNet → 3D UNet blocks: uses blocks (evidence: `code/i2v_enhance/unet_i2vgen_xl.py`)
- 3D UNet blocks → Spatial transformer: uses transformer (evidence: `code/i2v_enhance/unet_3d_blocks.py`)
- 3D UNet blocks → Temporal transformer: uses temporal layers (evidence: `code/i2v_enhance/unet_3d_blocks.py`)
- Spatial transformer → Temporal transformer: uses temporal layers (evidence: `code/i2v_enhance/transformer_temporal.py`)
- Inference pipeline → Frame interpolation: initializes interpolation (evidence: `code/inference_i2v.py`)
- StreamingSVD trainer → Enhancement interface: enhances generated video (evidence: `README.md`)
- Enhancement interface → Frame interpolation: interpolates frames (evidence: `code/i2v_enhance/i2v_enhance_interface.py`)
- Inference pipeline → Result processing: processes results
- Result processing → Generated video: saves video
