How ssj9596/one-to-all-animation works
One-to-All Animation generates character videos from a reference image and motion/pose input, with 1.3B and 14B inference variants. The main user workflow prepares inputs, constructs a model pipeline, and generates video; pose estimation/retargeting and reference-conditioned models support motion control. The repository also provides dataset preparation, progressive training, and evaluation code. Inference wiring is supported by sampled scripts; training and evaluation links are limited to capabilities documented in the README because their runtime implementations were not sampled.
Main entry points: inference_1.3b.py 1.3B inference · inference_14b.py 14B inference · infer_preprocess.py Input preparation · dataset Training datasets
Components 20
Video generation
- Input preparation
video-generation/infer_preprocess.py - 1.3B inference
video-generation/inference_1.3b.py - 14B inference
video-generation/inference_14b.py - Text encoders
video-generation/opensora/encoder_variants - Wan transformers
video-generation/opensora/model_variants/wanx_diffusers_src - Video VAE
video-generation/opensora/vae_variants - Generated video
Pose and reference
- Pose extraction
video-generation/wanpose_utils/pose2d.py - Pose utilities
video-generation/wanpose_utils/pose2d_utils.py - Pose retargeting
video-generation/wanpose_utils/retarget_pose.py - DWPose detection
video-generation/dwpose_utils/dwpose_detector.py
Training data
- Training data prep
datasets/prepare_training_data.py - Pose pool prep
datasets/prepare_pose_pool.py - Training datasets
video-generation/opensora/dataset - Progressive training
video-generation/opensora/train/train.py - VAE training
video-generation/opensora/train/train_causalvae.py
Training and evaluation
- Evaluation metrics
video-generation/opensora/eval - Model checkpoints
checkpoints/download_checkpoints.py - Pretrained models
pretrained_models/download_pretrained_models.py
Other components
- Animation user
Connections 22
- Animation user → Input preparation: prepares inputs
README.md - Input preparation → 1.3B inference: supplies inputs
README.md - Input preparation → 14B inference: supplies inputs
README.md - 1.3B inference → Text encoders: loads encoder
video-generation/inference_1.3b.py - 1.3B inference → Wan transformers: builds transformer
video-generation/inference_1.3b.py - 1.3B inference → Video VAE: loads VAE
video-generation/inference_1.3b.py - 1.3B inference → Generated video: generates video
video-generation/inference_1.3b.py - 14B inference → Text encoders: loads encoder
video-generation/inference_14b.py - 14B inference → Wan transformers: builds transformer
video-generation/inference_14b.py - 14B inference → Video VAE: loads VAE
video-generation/inference_14b.py - 14B inference → Generated video: generates video
video-generation/inference_14b.py - Pose extraction → Pose utilities: uses utilities
video-generation/wanpose_utils/pose2d.py - Pose retargeting → Pose utilities: uses utilities
video-generation/wanpose_utils/retarget_pose.py - Animation user → Pose extraction: provides motion
README.md - Pose extraction → Input preparation: provides pose
- Training data prep → Training datasets: prepares data
README.md - Pose pool prep → Training datasets: prepares pose data
README.md - Training datasets → Progressive training: feeds training
README.md - Progressive training → Model checkpoints: produces weights
README.md - Pretrained models → Progressive training: initializes training
README.md - Training datasets → VAE training: feeds VAE training
README.md - Progressive training → Evaluation metrics: evaluated by
README.md