How lllyasviel/fooocus works
Purpose and Execution
Fooocus is a local/offline Python application for prompt- and image-guided SDXL image generation, exposed through a Gradio UI. Primary launch paths are entry_with_update.py (installation/update-aware launcher), launch.py (runtime bootstrap), and webui.py (UI assembly/server). Docker alternatives are Dockerfile, docker-compose.yml, and entrypoint.sh.
Startup, Arguments, and Configuration
CLI flags, hardware mode, networking, preset selection, paths, and precision policy enter through args_manager.py and are propagated into shared runtime state via shared.py. Configuration resolves model/output directories and defaults, while presets define model/style behavior. Core paths: modules/config.py, presets/default.json, modules/launch_util.py.
The process is designed to download missing model assets on demand; local model directories under models/ are the durable artifact boundary.
Browser UI and Request Boundary
webui.py constructs the Gradio interface, integrating custom UI behavior from modules/gradio_hijack.py, modules/ui_gradio_extensions.py, and browser assets in javascript/ and css/. It collects prompts, styles, generation settings, uploaded images/masks, and preset changes; it may bind locally, listen remotely, or use Gradio share endpoints. Optional basic authentication is handled by modules/auth.py.
Asynchronous Generation Orchestration
The UI submits work to a long-lived asynchronous worker rather than performing inference in request callbacks. modules/async_worker.py is the principal queue/control boundary between interactive Gradio events and GPU-intensive execution. It coordinates request modes—text-to-image, image variation/upscale, inpaint/outpaint, and image-prompt conditioning—then returns previews/results to the UI. modules/private_logger.py owns result logging/output handling.
Prompt, Style, and Input Preparation
Before diffusion, requests are expanded and normalized: styles and SDXL prompt formatting are managed by modules/sdxl_styles.py; wildcard/inline prompt handling is supported by modules/util.py; Fooocus’s model-backed prompt expansion is in extras/expansion.py. Image-driven operations prepare masks, crops, adapters, and preprocessing through modules/inpaint_worker.py, extras/inpaint_mask.py, and extras/preprocessors.py.
This stage converts user-facing inputs into prompts, latent/image conditions, masks, and model selections consumed by the pipeline.
Diffusion Pipeline and Model Lifecycle
modules/default_pipeline.py is the application-level SDXL pipeline, with model loading and selection through modules/model_loader.py and modules/core.py. It bridges application semantics to the vendored/patched inference engine in ldm_patched/, including checkpoint loading, LoRA application, sampling, ControlNet, and VRAM management. Key engine paths: ldm_patched/modules/sd.py, ldm_patched/modules/sample.py, ldm_patched/modules/model_management.py.
The primary flow is: prepared request → load/patch base/refiner/LoRA/control models as needed → encode conditions and initialize latents → iterative sampler execution → decode image → return/save output.
Extensions, Safety, and Deployment Constraints
Optional enrichment and postprocessing are local ML integrations: image interrogation/captioning (extras/interrogate.py, vendored extras/BLIP/), face processing (extras/face_crop.py), image-prompt adapters (extras/ip_adapter.py), and safety checking (extras/safety_checker/models/safety_checker.py). Grounding and segmentation assets are vendored under extras/GroundingDINO/ and extras/sam/.
Runtime is Python/PyTorch with CUDA as the primary target; DirectML, ROCm, MPS, and CPU modes are supported through launch/runtime options. Model downloads rely on Hugging Face-hosted artifacts; public sharing relies on Gradio’s .gradio.live service.
Main entry points: webui.py Gradio Web UI · default_pipeline.py SDXL application pipeline · async_worker.py Async generation worker · config.py Configuration
Components 18
Startup & Configuration
- Launchers · Update-aware and runtime bootstrap paths start the application.
entry_with_update.py - Arguments & shared state · CLI, hardware, network, preset, and precision choices become shared runtime state.
args_manager.py - Configuration · Resolves model/output locations and defaults.
modules/config.py - Default preset · Defines default model and style behavior.
presets/default.json
Interactive UI
- Gradio Web UI · Collects prompts, settings, images, masks, and preset changes.
webui.py - UI customizations · Custom Gradio behavior and browser assets extend the interface.
modules/gradio_hijack.py - Basic authentication · Optional authentication for the UI server.
modules/auth.py
Request Preparation
- Async generation worker · Long-lived control boundary between UI events and GPU execution.
modules/async_worker.py - Prompt & style preparation · Expands, normalizes, and formats prompts and styles.
modules/sdxl_styles.py - Image input preparation · Builds masks, crops, adapters, and preprocessed image conditions.
modules/inpaint_worker.py
Inference Runtime
- SDXL application pipeline · Selects models, encodes conditions, samples latents, and decodes images.
modules/default_pipeline.py - Model lifecycle · Loads and selects checkpoints, refiners, LoRAs, and control models.
modules/model_loader.py - Patched inference engine · Provides checkpoint loading, sampling, ControlNet, and VRAM management.
Artifacts & Integrations
- Local model store · Persistent boundary for downloaded model assets.
- Output logging · Saves and logs generated results.
modules/private_logger.py - Optional ML enrichments · Captioning, face processing, IP adapters, safety, grounding, and segmentation.
extras/interrogate.py - Hugging Face artifacts · Supplies missing model assets on demand.
- Gradio share service · Optional public UI endpoint.
Connections 20
- Launchers → Arguments & shared state: parses runtime options
- Arguments & shared state → Configuration: configures
- Default preset → Configuration: provides defaults
- Configuration → Gradio Web UI: initializes
- Gradio Web UI → UI customizations: extends with
- Basic authentication → Gradio Web UI: protects
- Gradio Web UI → Async generation worker: submits generation jobs
- Gradio Web UI → Gradio share service: optionally publishes through
- Async generation worker → Prompt & style preparation: prepares prompts
- Async generation worker → Image input preparation: prepares image conditions
- Prompt & style preparation → SDXL application pipeline: conditions request
- Image input preparation → SDXL application pipeline: supplies masks and adapters
- Async generation worker → SDXL application pipeline: orchestrates modes
- SDXL application pipeline → Model lifecycle: selects required models
- Model lifecycle → Local model store: loads local assets
- Hugging Face artifacts → Local model store: downloads missing assets
- SDXL application pipeline → Patched inference engine: executes inference
- Optional ML enrichments → Async generation worker: optional enrichment
- SDXL application pipeline → Output logging: returns generated images
- Output logging → Gradio Web UI: streams previews and results