How zhexinliang/control-color works
Project type and purpose
• This is an open-source, full-pipeline image colorization tool built on a latent diffusion framework.
• It provides an interactive Gradio demo (“test.py”) that lets users colorize grayscale images via region selection, stroke painting, or text prompts.
• Under the hood it leverages:
– A pretrained VQGAN/VAE (“taming” modules) for image encoding/decoding
– A latent diffusion model (LDM) core (“ldm” modules) for sampling denoised latents
– A ControlLDM extension (“cldm” modules) that injects user controls into the diffusion process
– Annotator utilities (“annotator”) to parse user input (masks, strokes)
Top-level components (per file tree)
• Config & model definitions
– CtrlColor_environ.yaml
– config.py
– models/*.yaml (model hyperparameters)
• Pretrained model storage: pretrained_models/
• Annotator: annotator/util.py
• Autoencoder (taming): taming/modules/autoencoder & taming/models/vqgan.py
• Diffusion engine (LDM core): ldm/models/diffusion & ldm/modules
• Control extension: cldm/cldm.py & auxiliary sampling hacks
• Orchestration & UI: share.py, test.py
Key architectural patterns
• Layered pipeline:
1) Input handling (Gradio UI)
2) Pre-processing & annotation (masks, strokes)
3) Latent encoding (taming autoencoder)
4) Controlled diffusion sampling (LDM + ControlLDM)
5) Latent decoding back to image
• Configuration-driven instantiation: YAML files + config.py
• Modular separation:
– “annotator” is orthogonal to diffusion core
– “taming” provides VAE
– “ldm” implements diffusion backbone
– “cldm” injects control into the diffusion loop