How aleafy/relightvid works
Project type and purpose
• This is an open-source, CLI-driven Python framework for temporally consistent video relighting using diffusion models. It is not a web or full-stack app but a research-oriented tool that assembles pre-trained models into a pipeline.
File structure
• Root
• inference.py ← CLI entry point, orchestrates everything
• configs/inference_fbc.yaml ← runtime parameters
• requirements.txt
• assets/ ← sample videos, masks, backgrounds
• pipelines/
• diffusion.py ← diffusion scheduler & sampler
• relightvid.py ← relighting workflow
• inference.py ← pipeline driver called by CLI
• modules/
• kl_autoencoder/autoencoder.py
• vqvae/autoencoder.py & model.py
• openclip/modules.py ← embedding/text conditioning
• video_unet_temporal/… ← temporal U-Net, attention & motion modules
• utils/
• video_utils.py, image_utils.py ← I/O & preprocessing
• load_utils.py, model_utils.py ← weight loading & model construction
README for additional insights
• Inputs: original video, per-frame masks, background video (or text/HDR in the future)
• Pre-trained models: RealisticVision-v51 (diffusion base), IC-Light, RelightVid weights (motion modules)
• Key emphasis on frame-to-frame consistency via temporal modules