How myniuuu/anicrafter works
AniCrafter creates human-centric animated video by conditioning video diffusion on a rendered 3D Gaussian-splat avatar and a background scene. The documented workflow reconstructs an avatar from a character image, animates it using SMPLX motion, combines its renderings with a background, then generates and saves the result. For in-the-wild replacement, preprocessing adds human-mask parsing, pose/SMPLX estimation, and background inpainting. The README documents these stages; excerpts show pipeline setup and rendering-related code but do not expose every orchestration call, so uncertain wiring is omitted.
Main entry points: wan_video_v2v_combine.py AniCrafter diffusion · run_pipeline_with_preprocess.py Replacement entry · run_pipeline.py Avatar-background condition · model_manager.py Model loading
Components 17
Video preprocessing
- Human masks
engine/BiRefNet/inference.py - Pose and SMPLX
engine/pose_estimation/pose_estimator.py - Background inpainting
ProPainter/inference_propainter.py
Avatar reconstruction
- Avatar reconstruction
LHM/runners/infer/human_lrm.py
Motion and conditioning
- SMPLX model
LHM/models/rendering/smpl_x.py - Animated avatar render
LHM/models/rendering/synthesizer.py - Avatar-background condition
run_pipeline.py
Video generation
- AniCrafter diffusion
diffsynth/pipelines/wan_video_v2v_combine.py - Video diffusion models
diffsynth/models/wan_video_dit.py
Inference runtime
- Model loading
diffsynth/models/model_manager.py - Single-GPU entry
run_pipeline.py - USP entry
run_pipeline_usp.py - Replacement entry
run_pipeline_with_preprocess.py - Rendered video output
run_pipeline.py
Other components
- User
- Character image
- Scene or video
Connections 25
- User → Character image: provides
- User → Scene or video: provides
- Scene or video → Human masks: parse masks
- Scene or video → Pose and SMPLX: estimate motion
- Scene or video → Background inpainting: remove person
- Character image → Avatar reconstruction: reconstructs
- Avatar reconstruction → Animated avatar render: provides avatar
- Pose and SMPLX → SMPLX model: provides motion
- SMPLX model → Animated avatar render: drives animation
- Animated avatar render → Avatar-background condition: supplies renderings
- Scene or video → Avatar-background condition: supplies background
- Background inpainting → Avatar-background condition: supplies clean background
- Avatar-background condition → AniCrafter diffusion: conditions
- Model loading → AniCrafter diffusion: loads models for
- AniCrafter diffusion → Video diffusion models: runs diffusion models
- Single-GPU entry → Model loading: loads
- Single-GPU entry → AniCrafter diffusion: invokes
- USP entry → Model loading: loads
- USP entry → AniCrafter diffusion: invokes with USP
- Replacement entry → Human masks: invokes
- Replacement entry → Pose and SMPLX: invokes
- Replacement entry → Background inpainting: invokes
- Replacement entry → Model loading: loads
- Replacement entry → AniCrafter diffusion: invokes
- AniCrafter diffusion → Rendered video output: generates