How guanyimeng/sd_svd_backend works
Project Type and Purpose
• This is a Python-based backend (CLI‑driven) for running three image‑generation models: Stable Diffusion 1.5 (SD), Stable Diffusion XL (SDXL), and a custom SVD model.
• It provides script entry points that accept either text prompts (SD & SDXL) or input images (SVD), then produce images or videos.
File Structure
• Top level:
• scripts/ : CLI entry points (SD_run.py, SDXL_run.py, SVD_run.py)
• madhouse/ : Python package containing three subpackages (SD, SDXL, SVD), each with model‑wrapping functions (e.g., SD_text2image.py)
• config/config.ini : runtime settings (e.g., model paths, inference parameters)
• inputs/ : optional source images for SVD
• outputs/ : where generated media are saved
• models/Put_models_here : placeholder for local model weights
• requirements.txt : dependency declarations (torch, huggingface_hub, others)
README Insights
• Prereqs: PyTorch install, requirements.txt, huggingface-cli login for model access.
• Execution: set PYTHONPATH, then run one of the scripts with a prompt or input file path.
Main Components
a. Entry‑Point Layer (scripts/)
• SD_run.py
• SDXL_run.py
• SVD_run.py
b. Configuration Loader
• Reads config/config.ini
c. Model Modules (madhouse/)
• madhouse.SD.SD_text2image
• madhouse.SDXL.SXDL_text2image
• madhouse.SVD.SVD_func
d. Model Weights
• Local storage (models/) or remote on Huggingface Hub
e. Inference Engine
• PyTorch runtime (GPU/CPU)
f. I/O Storage
• inputs/ (for SVD)
• outputs/ (images, videos)
g. External Services
• Huggingface Hub (model download, authentication)
Relationships and Data Flow
1. User invokes a script with CLI arguments.
2. Script loads config via the Configuration Loader.
3. Script calls the appropriate Model Module function.
4. Model Module checks for local weights (models/) or fetches from Huggingface Hub (via huggingface-cli+API).
5. PyTorch environment runs inference:
– SD & SDXL: text → latent → image
– SVD: input image → transform → output
6. Generated output is written to outputs/
Architectural Patterns & Design Principles
• Layered design: CLI layer → config → model wrapper → inference engine → storage.
• Single responsibility: separate scripts for each model, separate modules per model type.
• Extensibility: new models can be dropped into madhouse/ and new scripts can be added.