# portfoliogrowthsolutions/stablediffusion architecture

> Architecture diagram of the GitHub repository portfoliogrowthsolutions/stablediffusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/portfoliogrowthsolutions/stablediffusion
- Repository: https://github.com/portfoliogrowthsolutions/stablediffusion

## Overview

##### Project type and purpose
- Full-stack Python web application with a rich JavaScript/HTML/CSS front-end (built on Gradio), and a Python back-end that loads and runs Stable Diffusion and related models.
- Purpose: provide an interactive browser UI for text-to-image (and related) workflows using local ML models.

##### Main components
A. Front-end (Client/UI)
- Gradio UI server (auto-generated HTML forms & JS)
- Static assets:
- javascript/ directory (UI helpers, context menus, drag-drop, token counters, etc.)
- html/ directory (panels, cards, buttons templates)
- style.css, script.js
- Browser (the user’s web browser where the UI is rendered)

B. Back-end (Server/API & Business Logic)
- Entry-point scripts:
- webui.py / launch.py
- modules/launch_utils.py (parses CLI, env setup)
- API layer:
- modules/api/api.py (defines HTTP endpoints for image generation, settings, extensions)
- modules/api/models.py (data schemas, pydantic models)
- Core processing pipeline:
- modules/processing.py, modules/txt2img.py, modules/img2img.py
- modules/models/: model loader & wrapper classes (sd_unet, sd_vae, schedulers, samplers)
- modules/postprocessing.py (saving images, embedding metadata)
- State & configuration:
- modules/config_states.py, modules/options.py, modules/shared_state.py
- configs/ (YAML presets for various inference modes)
- Extension/plugin system:
- extensions-builtin/ (bundled extensions: Lora, LDSR, canvas-zoom, hypertile…)
- modules/extensions.py, modules/script_callbacks.py (hooks/events)
- extensions/ (user-provided plugins)

C. File system & storage
- models/ (Stable Diffusion checkpoints, VAE, DeepBooru, Karlo)
- embeddings/ (Textual-inversion embeddings)
- textual_inversion_templates/
- localizations/ (language packs)
- outputs folder (images saved by processing pipeline)

D. External dependencies & services
- PyTorch / CUDA / ROCm (GPU/CPU computation back-end)
- Optional libraries: xformers, ONNX, TensorRT adapters (via modules/xpu_specific.py, modules/npu_specific.py)
- ngrok tunnel (modules/ngrok.py) for remote access
- Third-party models: GFPGAN, CodeFormer, RealESRGAN, SwinIR, etc. (invoked through modules/*_model.py)

##### Relationships & Data flow
1. User interacts with Browser → Gradio UI (HTTP/WebSocket)
2. Gradio UI → Python API endpoints (modules/api/api.py)
3. API → Core processing pipeline (modules/processing.py → txt2img/img2img modules)
4. Processing → Model loader & sampler → GPU/CPU → generate tensor outputs
5. Outputs → Postprocessing → save image to disk + embed metadata
6. Postprocessing → Return image bytes + metadata → API → UI → Browser display
7. Extension events: on_model_loaded, on_image_saved → trigger extension hooks

##### Architectural patterns & design principles
- MVC-style separation:
– View: Gradio UI + static assets
– Controller/API: modules/api & launch_utils
– Model: ML models + business logic in modules/processing
- Plugin architecture: script_callbacks, dynamic import of extensions, clear hook points
- Singleton/Shared state: modules/shared.py manages global options, themes, loaded checkpoints
- Config-driven: YAML files in configs/ drive inference presets; pydantic validation

##### Relevant technologies/frameworks
- Python 3.10+, PyTorch (torch), xformers, diffusers (optional)
- Gradio for UI & HTTP/WebSocket server
- JavaScript (vanilla + Gradio extensions) for interactive frontend
- YAML for configuration
- GitHub Actions for CI (see .github/workflows)
- pytest for tests (test/ directory)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Front-end Layer
    subgraph "Front-end (Client/UI)"
        direction TB
        Browser["Web Browser"]:::frontend
        Gradio["Gradio UI Server"]:::frontend
        subgraph "Static Assets"
            direction TB
            HTML["html/"]:::frontend
            JS["javascript/"]:::frontend
            StyleCSS["style.css"]:::frontend
            ScriptJS["script.js"]:::frontend
        end
    end

    %% Back-end Core
    subgraph "Back-end Core (Server/API & Business Logic)"
        direction TB
        subgraph "Entry-Point Scripts"
            direction TB
            WebUI["webui.py"]:::backend
            Launch["launch.py"]:::backend
            LaunchUtils["modules/launch_utils.py"]:::backend
        end
        subgraph "API Layer"
            direction TB
            API["modules/api/api.py"]:::backend
            Models["modules/api/models.py"]:::backend
        end
        subgraph "Processing Pipeline"
            direction TB
            Processing["modules/processing.py"]:::backend
            Txt2Img["modules/txt2img.py"]:::backend
            Img2Img["modules/img2img.py"]:::backend
        end
        subgraph "Model Loading & Sampling"
            direction TB
            ModelLoader["modules/modelloader.py"]:::backend
            UNet["modules/sd_unet.py"]:::backend
            VAE["modules/sd_vae.py"]:::backend
        end
        PostProc["modules/postprocessing.py"]:::backend
        subgraph "State & Configuration"
            direction TB
            SharedState["modules/shared_state.py"]:::backend
            ConfigStates["modules/config_states.py"]:::backend
            Options["modules/options.py"]:::backend
            Presets["configs/"]:::backend
        end
    end

    %% Plugin System
    subgraph "Extensions / Plugin System"
        direction TB
        BuiltinExt["extensions-builtin/"]:::plugin
        UserExt["extensions/"]:::plugin
        ExtCore["modules/extensions.py"]:::plugin
        Callbacks["modules/script_callbacks.py"]:::plugin
    end

    %% File System & Storage
    subgraph "File System & Storage"
        direction TB
        ModelsFS["models/"]:::storage
        EmbeddingsFS["embeddings/"]:::storage
        TI_Templates["textual_inversion_templates/"]:::storage
        LocalizationsFS["localizations/"]:::storage
        Outputs["outputs/"]:::storage
    end

    %% External Dependencies & Services
    subgraph "External Dependencies & Services"
        direction TB
        PyTorch["PyTorch / CUDA / ROCm"]:::external
        XPU["modules/xpu_specific.py"]:::external
        NPU["modules/npu_specific.py"]:::external
        NGROK["modules/ngrok.py"]:::external
        ThirdParty["Third-party models (GFPGAN, RealESRGAN, etc.)"]:::external
    end

    %% Connections
    Browser -->|"loads UI"| Gradio
    Gradio -->|"HTTP/WebSocket"| API
    API -->|"invoke"| Processing
    Processing -->|"txt2img/img2img"| Txt2Img
    Processing -->|"txt2img/img2img"| Img2Img
    Txt2Img -->|"load model"| ModelLoader
    Img2Img -->|"load model"| ModelLoader
    ModelLoader -->|"instantiate"| UNet
    ModelLoader -->|"instantiate"| VAE
    Processing -->|"results"| PostProc
    PostProc -->|"save image"| Outputs
    PostProc -->|"return bytes"| API
    API -->|"HTTP response"| Gradio
    UNet -->|"on_model_loaded"| Callbacks
    VAE -->|"on_model_loaded"| Callbacks
    PostProc -->|"on_image_saved"| Callbacks
    ExtCore -->|"register hooks"| Callbacks
    Presets -->|"load presets"| Processing
    SharedState -->|"global options"| Processing
    SharedState -->|"global options"| API
    PyTorch -->|"compute"| ModelLoader
    XPU -->|"accelerate"| ModelLoader
    NPU -->|"accelerate"| ModelLoader
    NGROK -->|"tunnel"| API
    ThirdParty -->|"external model call"| Processing

    %% Click Events
    click HTML "https://github.com/portfoliogrowthsolutions/stablediffusion/tree/master/html/"
    click JS "https://github.com/portfoliogrowthsolutions/stablediffusion/tree/master/javascript/"
    click StyleCSS "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/style.css"
    click ScriptJS "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/script.js"
    click WebUI "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/webui.py"
    click Launch "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/launch.py"
    click LaunchUtils "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/launch_utils.py"
    click API "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/api/api.py"
    click Models "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/api/models.py"
    click Processing "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/processing.py"
    click Txt2Img "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/txt2img.py"
    click Img2Img "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/img2img.py"
    click ModelLoader "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/modelloader.py"
    click UNet "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/sd_unet.py"
    click VAE "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/sd_vae.py"
    click PostProc "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/postprocessing.py"
    click SharedState "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/shared_state.py"
    click ConfigStates "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/config_states.py"
    click Options "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/options.py"
    click Presets "https://github.com/portfoliogrowthsolutions/stablediffusion/tree/master/configs/"
    click BuiltinExt "https://github.com/portfoliogrowthsolutions/stablediffusion/tree/master/extensions-builtin/"
    click UserExt "https://github.com/portfoliogrowthsolutions/stablediffusion/tree/master/extensions/"
    click ExtCore "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/extensions.py"
    click Callbacks "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/script_callbacks.py"
    click XPU "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/xpu_specific.py"
    click NPU "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/npu_specific.py"
    click NGROK "https://github.com/portfoliogrowthsolutions/stablediffusion/blob/master/modules/ngrok.py"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#0366d6,color:#000;
    classDef backend fill:#DFFFE0,stroke:#228B22,color:#000;
    classDef plugin fill:#FFEFD5,stroke:#FF8C00,color:#000;
    classDef external fill:#E8E8E8,stroke:#696969,color:#000;
    classDef storage fill:#FFFACD,stroke:#DAA520,color:#000;
```
