# shootthesound/comfyui-realtime-lora architecture

> Architecture diagram of the GitHub repository shootthesound/comfyui-realtime-lora, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shootthesound/comfyui-realtime-lora
- Repository: https://github.com/shootthesound/comfyui-realtime-lora

## Overview

This ComfyUI custom-node toolkit trains LoRAs, analyzes and selectively applies them, and edits model layers and selected model components. The main workflow starts in ComfyUI: a user supplies training data and settings, a trainer invokes a chosen backend and produces a LoRA; users can then analyze/load LoRAs or edit model components in generation workflows. The README documents capabilities beyond the sampled excerpts; those branches are included, but only evidenced relationships are drawn.

Main entry points: [`sdxl_lora_trainer.py`](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/sdxl_lora_trainer.py) (sd-scripts trainers), [`musubi_zimage_lora_trainer.py`](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/musubi_zimage_lora_trainer.py) (Musubi trainers), [`lora_analyzer_v2.py`](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/lora_analyzer_v2.py) (Combined analyzer), [`realtime_lora_trainer.py`](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/realtime_lora_trainer.py) (AI-Toolkit trainer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["LoRA training"]
  node_ai_trainer["AI-Toolkit trainer"]
  node_sd_trainer["sd-scripts trainers"]
  node_musubi_trainers["Musubi trainers"]
  node_musubi_config["Musubi dataset config"]
  node_sd_config["sd-scripts config"]
end

subgraph group_loading["LoRA analysis and loading"]
  node_analyzer_v1["LoRA analyzer<br/>[lora_analyzer.py]"]
  node_analyzer_v2["Combined analyzer"]
  node_quant["Quantization helpers<br/>[quant_utils.py]"]
  node_selective["Selective loaders"]
  node_scheduled["Scheduled loader"]
end

subgraph group_editing["Model editing"]
  node_layer_editor["Model layer editor"]
  node_klein_editor["FLUX Klein editor"]
  node_vae_editor["FLUX VAE editor"]
  node_zimage_editor["Z-Image editor"]
  node_qwen4_editor["Qwen3 4B editor"]
  node_qwen8_editor["Qwen3 8B editor"]
end

subgraph group_inspection["Encoder inspection"]
  node_vae_inspector["FLUX VAE inspector"]
  node_qwen4_inspector["Qwen3 4B inspector"]
  node_qwen8_inspector["Qwen3 8B inspector"]
end

node_user(("ComfyUI user"))
node_comfy["ComfyUI workflows"]
node_ai_backend["AI-Toolkit"]
node_sd_backend["sd-scripts"]
node_musubi_backend["Musubi Tuner"]

node_user -->|"builds workflows"| node_comfy
node_comfy -->|"starts training"| node_ai_trainer
node_comfy -->|"starts training"| node_sd_trainer
node_comfy -->|"starts training"| node_musubi_trainers
node_ai_trainer -->|"invokes backend"| node_ai_backend
node_sd_trainer -->|"invokes backend"| node_sd_backend
node_musubi_trainers -->|"invokes backend"| node_musubi_backend
node_musubi_trainers -->|"imports config"| node_musubi_config
node_sd_trainer -->|"imports config"| node_sd_config
node_comfy -->|"analyzes LoRAs"| node_analyzer_v1
node_comfy -->|"analyzes LoRAs"| node_analyzer_v2
node_comfy -->|"loads LoRAs"| node_selective
node_comfy -->|"schedules strength"| node_scheduled
node_analyzer_v2 -->|"uses helpers"| node_quant
node_analyzer_v2 -->|"imports loader"| node_selective
node_comfy -->|"edits model layers"| node_layer_editor
node_comfy -->|"edits FLUX Klein"| node_klein_editor
node_comfy -->|"edits VAE"| node_vae_editor
node_comfy -->|"edits Z-Image"| node_zimage_editor
node_comfy -->|"edits encoder"| node_qwen4_editor
node_comfy -->|"edits encoder"| node_qwen8_editor
node_comfy -->|"inspects VAE"| node_vae_inspector
node_comfy -->|"inspects encoder"| node_qwen4_inspector
node_comfy -->|"inspects encoder"| node_qwen8_inspector

click node_ai_trainer "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/realtime_lora_trainer.py"
click node_sd_trainer "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/sdxl_lora_trainer.py"
click node_musubi_trainers "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/musubi_zimage_lora_trainer.py"
click node_musubi_config "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/musubi_zimage_config_template.py"
click node_sd_config "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/sdxl_config_template.py"
click node_analyzer_v1 "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/lora_analyzer.py"
click node_analyzer_v2 "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/lora_analyzer_v2.py"
click node_quant "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/quant_utils.py"
click node_selective "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/selective_lora_loader.py"
click node_scheduled "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/scheduled_lora_loader.py"
click node_layer_editor "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/model_layer_analyzer.py"
click node_klein_editor "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/flux_klein_debiaser_node.py"
click node_vae_editor "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/flux_vae_debiaser_node.py"
click node_zimage_editor "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/zimage_deep_debiaser_node.py"
click node_qwen4_editor "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_4b_text_encoder_debiaser_node.py"
click node_qwen8_editor "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_8b_text_encoder_debiaser_node.py"
click node_vae_inspector "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/flux_vae_inspector_node.py"
click node_qwen4_inspector "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_4b_text_encoder_inspector_node.py"
click node_qwen8_inspector "https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_8b_text_encoder_inspector_node.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ai_trainer,node_sd_trainer,node_musubi_trainers,node_musubi_config,node_sd_config,node_user,node_comfy toneBlue
class node_analyzer_v1,node_analyzer_v2,node_quant,node_selective,node_scheduled toneAmber
class node_layer_editor,node_klein_editor,node_vae_editor,node_zimage_editor,node_qwen4_editor,node_qwen8_editor toneMint
class node_vae_inspector,node_qwen4_inspector,node_qwen8_inspector toneRose
class node_ai_backend,node_sd_backend,node_musubi_backend toneIndigo
```

## Components

### LoRA training

- [AI-Toolkit trainer](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/realtime_lora_trainer.py): `realtime_lora_trainer.py`
- [sd-scripts trainers](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/sdxl_lora_trainer.py): `sdxl_lora_trainer.py`
- [Musubi trainers](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/musubi_zimage_lora_trainer.py): `musubi_zimage_lora_trainer.py`
- [Musubi dataset config](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/musubi_zimage_config_template.py): `musubi_zimage_config_template.py`
- [sd-scripts config](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/sdxl_config_template.py): `sdxl_config_template.py`

### LoRA analysis and loading

- [LoRA analyzer](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/lora_analyzer.py): `lora_analyzer.py`
- [Combined analyzer](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/lora_analyzer_v2.py): `lora_analyzer_v2.py`
- [Quantization helpers](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/quant_utils.py): `quant_utils.py`
- [Selective loaders](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/selective_lora_loader.py): `selective_lora_loader.py`
- [Scheduled loader](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/scheduled_lora_loader.py): `scheduled_lora_loader.py`

### Model editing

- [Model layer editor](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/model_layer_analyzer.py): `model_layer_analyzer.py`
- [FLUX Klein editor](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/flux_klein_debiaser_node.py): `flux_klein_debiaser_node.py`
- [FLUX VAE editor](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/flux_vae_debiaser_node.py): `flux_vae_debiaser_node.py`
- [Z-Image editor](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/zimage_deep_debiaser_node.py): `zimage_deep_debiaser_node.py`
- [Qwen3 4B editor](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_4b_text_encoder_debiaser_node.py): `qwen3_4b_text_encoder_debiaser_node.py`
- [Qwen3 8B editor](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_8b_text_encoder_debiaser_node.py): `qwen3_8b_text_encoder_debiaser_node.py`

### Encoder inspection

- [FLUX VAE inspector](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/flux_vae_inspector_node.py): `flux_vae_inspector_node.py`
- [Qwen3 4B inspector](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_4b_text_encoder_inspector_node.py): `qwen3_4b_text_encoder_inspector_node.py`
- [Qwen3 8B inspector](https://github.com/shootthesound/comfyui-realtime-lora/blob/main/qwen3_8b_text_encoder_inspector_node.py): `qwen3_8b_text_encoder_inspector_node.py`

### Other components

- **ComfyUI user**
- **ComfyUI workflows**
- **AI-Toolkit**
- **sd-scripts**
- **Musubi Tuner**

## Connections

- ComfyUI user → ComfyUI workflows: builds workflows (evidence: `README.md`)
- ComfyUI workflows → AI-Toolkit trainer: starts training (evidence: `README.md`)
- ComfyUI workflows → sd-scripts trainers: starts training (evidence: `README.md`)
- ComfyUI workflows → Musubi trainers: starts training (evidence: `README.md`)
- AI-Toolkit trainer → AI-Toolkit: invokes backend (evidence: `README.md`)
- sd-scripts trainers → sd-scripts: invokes backend (evidence: `README.md`)
- Musubi trainers → Musubi Tuner: invokes backend (evidence: `musubi_zimage_lora_trainer.py`)
- Musubi trainers → Musubi dataset config: imports config (evidence: `musubi_zimage_lora_trainer.py`)
- sd-scripts trainers → sd-scripts config: imports config (evidence: `sdxl_lora_trainer.py`)
- ComfyUI workflows → LoRA analyzer: analyzes LoRAs (evidence: `README.md`)
- ComfyUI workflows → Combined analyzer: analyzes LoRAs (evidence: `README.md`)
- ComfyUI workflows → Selective loaders: loads LoRAs (evidence: `README.md`)
- ComfyUI workflows → Scheduled loader: schedules strength (evidence: `README.md`)
- Combined analyzer → Quantization helpers: uses helpers (evidence: `lora_analyzer_v2.py`)
- Combined analyzer → Selective loaders: imports loader (evidence: `lora_analyzer_v2.py`)
- ComfyUI workflows → Model layer editor: edits model layers (evidence: `README.md`)
- ComfyUI workflows → FLUX Klein editor: edits FLUX Klein (evidence: `README.md`)
- ComfyUI workflows → FLUX VAE editor: edits VAE (evidence: `README.md`)
- ComfyUI workflows → Z-Image editor: edits Z-Image
- ComfyUI workflows → Qwen3 4B editor: edits encoder
- ComfyUI workflows → Qwen3 8B editor: edits encoder
- ComfyUI workflows → FLUX VAE inspector: inspects VAE (evidence: `README.md`)
- ComfyUI workflows → Qwen3 4B inspector: inspects encoder (evidence: `README.md`)
- ComfyUI workflows → Qwen3 8B inspector: inspects encoder (evidence: `README.md`)
