How aigc-apps/easyanimate works
EasyAnimate is a high-resolution, long-video generation and model-training system. Its primary workflow accepts prompts and optional image, video, or control inputs through the UI/API or ComfyUI, runs a generation pipeline, and returns generated video. Separate branches cover dataset preparation and captioning, DiT/LoRA and VAE training, control/inpainting, and reward-based optimization. The README documents these capabilities; some branches are represented at their directory or entry-script level because their implementation was not sampled.
Main entry points: pipeline_easyanimate.py Generation pipeline · ui.py Gradio UI · transformer3d.py Video transformer · comfyui_nodes.py ComfyUI nodes
Components 21
User interfaces
- App launcher
app.py - Gradio UI
easyanimate/ui/ui.py - Inference API
easyanimate/api/api.py - ComfyUI nodes
comfyui/comfyui_nodes.py
Video generation
- Generation pipeline
easyanimate/pipeline/pipeline_easyanimate.py - Control pipeline
easyanimate/pipeline/pipeline_easyanimate_control.py - Inpaint pipeline
easyanimate/pipeline/pipeline_easyanimate_inpaint.py - Video transformer
easyanimate/models/transformer3d.py - Attention blocks
easyanimate/models/attention.py - Video autoencoder
easyanimate/models/autoencoder_magvit.py - Causal 3D encoder
easyanimate/vae/ldm/models/omnigen_enc_dec.py - Diffusion utilities
easyanimate/utils/gaussian_diffusion.py
Training and data
- Video datasets
easyanimate/data/dataset_image_video.py - Bucket sampling
easyanimate/data/bucket_sampler.py - DiT training
scripts/train.py - VAE training
scripts/train_vae.py - Caption preprocessing
easyanimate/video_caption - LoRA utilities
easyanimate/utils/lora_utils.py
Evaluation and reward
- Reward scoring
easyanimate/reward/reward_fn.py
Other components
- User
- Generated video
Connections 27
- User → Gradio UI: submits request
- App launcher → Gradio UI: launches
app.py - App launcher → Inference API: registers routes
app.py - User → ComfyUI nodes: configures workflow
- Gradio UI → Generation pipeline: runs inference
easyanimate/ui/ui.py - Gradio UI → Control pipeline: runs controls
easyanimate/ui/ui.py - Gradio UI → Inpaint pipeline: runs inpainting
easyanimate/ui/ui.py - Inference API → Generation pipeline: runs inference
- ComfyUI nodes → Generation pipeline: runs generation
comfyui/comfyui_nodes.py - ComfyUI nodes → Control pipeline: runs controls
comfyui/comfyui_nodes.py - ComfyUI nodes → Inpaint pipeline: runs inpainting
comfyui/comfyui_nodes.py - Generation pipeline → Video transformer: denoises latents
easyanimate/pipeline/pipeline_easyanimate.py - Generation pipeline → Video autoencoder: decodes latents
- Control pipeline → Video transformer: denoises controls
easyanimate/pipeline/pipeline_easyanimate_control.py - Inpaint pipeline → Video transformer: denoises masked input
easyanimate/pipeline/pipeline_easyanimate_inpaint.py - Video transformer → Attention blocks: uses blocks
easyanimate/models/transformer3d.py - Video autoencoder → Causal 3D encoder: uses encoder
easyanimate/models/autoencoder_magvit.py - Generation pipeline → Diffusion utilities: uses sampling utilities
- Generation pipeline → Generated video: returns video
- Video datasets → Bucket sampling: batches by bucket
- Video datasets → DiT training: supplies training data
- Bucket sampling → DiT training: provides batches
- DiT training → Video transformer: trains model
- VAE training → Video autoencoder: trains autoencoder
- Caption preprocessing → Video datasets: prepares metadata
- DiT training → LoRA utilities: supports LoRA training
- Reward scoring → LoRA utilities: scores optimization