# oobabooga/text-generation-webui architecture

> Architecture diagram of the GitHub repository oobabooga/text-generation-webui, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oobabooga/text-generation-webui
- Repository: https://github.com/oobabooga/text-generation-webui

## Overview

##### Main Components

a) Frontend Layer:
- Gradio Web Interface (modules/ui.py and related files)
- Multiple UI tabs (Chat, Default, Notebook, Parameters, Model, Training, Session)
- Static assets (CSS/JS files in css/ and js/ directories)

b) Backend Core:
- Server component (server.py)
- Extension system (modules/extensions.py)
- Model management (modules/models.py)
- Text generation pipeline (modules/text_generation.py)

c) Model Loaders:
- Multiple loader implementations:
- Transformers
- llama.cpp
- ExLlamaV2
- AutoGPTQ
- TensorRT-LLM
- Others (HQQ, AQLM)

d) Extensions System:
- Core extensions framework
- Built-in extensions (extensions/ directory):
- Training_PRO
- Multimodal
- OpenAI API compatibility
- Various TTS/STT solutions
- Character bias
- Others

e) Storage Layer:
- Model storage (models/ directory)
- LoRA storage (loras/ directory)
- Character templates (characters/ directory)
- Instruction templates (instruction-templates/ directory)

##### Relationships to Highlight

a) Data Flow:
- User input → Gradio UI → Server → Model Pipeline → Generated Text → UI
- API requests → OpenAI-compatible endpoints → Model Pipeline
- Training data → Training module → Model fine-tuning

b) Component Dependencies:
- Model loaders ↔ GPU/CPU resources
- Extensions ↔ Core server
- UI components ↔ Backend services

##### Specific Elements

a) Main Layers:
**[Web UI Layer (Gradio)]**
- ↓
**[Server Core Layer]**
- ↓
**[Model Management Layer]**
- ↓
**[Storage Layer]**

b) Extension System:
**[Extension Manager] → [Individual Extensions]**
- ↓
**[Core Services]**

c) Model Pipeline:
**[Model Loader Selection] → [Specific Model Implementation]**
- ↓
**[Text Generation Pipeline]**

##### Implementation Notes

- Version compatibility indicators
- Resource allocation paths
- Optional components (extensions)
- Security boundaries (API keys, authentication)

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Frontend["Frontend Layer"]
        UI[Main UI]:::frontend
        Chat[Chat UI]:::frontend
        Default[Default UI]:::frontend
        Notebook[Notebook UI]:::frontend
        Params[Parameters UI]:::frontend
        ModelMenu[Model Menu UI]:::frontend
        Session[Session UI]:::frontend
        Static[Static Assets]:::frontend
    end

    subgraph Core["Backend Core"]
        Server[Server Component]:::core
        ExtSystem[Extension System]:::core
        ModelMgmt[Model Management]:::core
        TextGen[Text Generation]:::core
        ChatLogic[Chat Logic]:::core
        Shared[Shared Utilities]:::core
    end

    subgraph ModelLoaders["Model Loaders"]
        GPTQ[AutoGPTQ]:::loader
        ExLlama[ExLlamaV2]:::loader
        LLaMaCPP[LLaMa CPP]:::loader
        TensorRT[TensorRT]:::loader
        GenLoaders[Generic Loaders]:::loader
    end

    subgraph Extensions["Extensions System"]
        Training[Training PRO]:::ext
        Multimodal[Multimodal]:::ext
        OpenAI[OpenAI API]:::ext
        TTS[Text-to-Speech]:::ext
        CharBias[Character Bias]:::ext
    end

    subgraph Storage["Storage Layer"]
        ModelsDir[(Models)]:::storage
        LoRADir[(LoRA)]:::storage
        CharsDir[(Characters)]:::storage
        InstTemp[(Instructions)]:::storage
        DatasetDir[(Datasets)]:::storage
    end

    subgraph Config["Configuration"]
        Settings[Settings Template]:::config
        CmdFlags[Command Flags]:::config
        ModelConfig[Model Config]:::config
        Presets[Presets]:::config
    end

    subgraph Training["Training Components"]
        TrainCore[Core Training]:::training
        TrainFormat[Training Formats]:::training
        LoraMgmt[LoRA Management]:::training
    end

    %% Relationships
    Frontend --> Server
    Server --> ModelMgmt
    ModelMgmt --> ModelLoaders
    ModelLoaders --> Storage
    ExtSystem --> Extensions
    Server --> TextGen
    TextGen --> ChatLogic
    Config --> Server
    Training --> ModelsDir
    Training --> LoRADir

    %% Click Events
    click UI "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui.py"
    click Chat "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui_chat.py"
    click Default "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui_default.py"
    click Notebook "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui_notebook.py"
    click Params "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui_parameters.py"
    click ModelMenu "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui_model_menu.py"
    click Session "https://github.com/oobabooga/text-generation-webui/blob/main/modules/ui_session.py"
    click Static "https://github.com/oobabooga/text-generation-webui/tree/main/css/"
    click Server "https://github.com/oobabooga/text-generation-webui/blob/main/server.py"
    click ExtSystem "https://github.com/oobabooga/text-generation-webui/blob/main/modules/extensions.py"
    click ModelMgmt "https://github.com/oobabooga/text-generation-webui/blob/main/modules/models.py"
    click TextGen "https://github.com/oobabooga/text-generation-webui/blob/main/modules/text_generation.py"
    click ChatLogic "https://github.com/oobabooga/text-generation-webui/blob/main/modules/chat.py"
    click Shared "https://github.com/oobabooga/text-generation-webui/blob/main/modules/shared.py"
    click GPTQ "https://github.com/oobabooga/text-generation-webui/blob/main/modules/AutoGPTQ_loader.py"
    click ExLlama "https://github.com/oobabooga/text-generation-webui/blob/main/modules/exllamav2.py"
    click LLaMaCPP "https://github.com/oobabooga/text-generation-webui/blob/main/modules/llamacpp_model.py"
    click TensorRT "https://github.com/oobabooga/text-generation-webui/blob/main/modules/tensorrt_llm.py"
    click GenLoaders "https://github.com/oobabooga/text-generation-webui/blob/main/modules/loaders.py"
    click Training "https://github.com/oobabooga/text-generation-webui/tree/main/extensions/Training_PRO/"
    click Multimodal "https://github.com/oobabooga/text-generation-webui/tree/main/extensions/multimodal/"
    click OpenAI "https://github.com/oobabooga/text-generation-webui/tree/main/extensions/openai/"
    click TTS "https://github.com/oobabooga/text-generation-webui/tree/main/extensions/silero_tts/"
    click CharBias "https://github.com/oobabooga/text-generation-webui/tree/main/extensions/character_bias/"
    click ModelsDir "https://github.com/oobabooga/text-generation-webui/tree/main/models/"
    click LoRADir "https://github.com/oobabooga/text-generation-webui/tree/main/loras/"
    click CharsDir "https://github.com/oobabooga/text-generation-webui/tree/main/characters/"
    click InstTemp "https://github.com/oobabooga/text-generation-webui/tree/main/instruction-templates/"
    click DatasetDir "https://github.com/oobabooga/text-generation-webui/tree/main/training/datasets/"
    click Settings "https://github.com/oobabooga/text-generation-webui/blob/main/settings-template.yaml"
    click CmdFlags "https://github.com/oobabooga/text-generation-webui/blob/main/CMD_FLAGS.txt"
    click ModelConfig "https://github.com/oobabooga/text-generation-webui/blob/main/models/config.yaml"
    click Presets "https://github.com/oobabooga/text-generation-webui/tree/main/presets/"
    click TrainCore "https://github.com/oobabooga/text-generation-webui/blob/main/modules/training.py"
    click TrainFormat "https://github.com/oobabooga/text-generation-webui/tree/main/training/formats/"
    click LoraMgmt "https://github.com/oobabooga/text-generation-webui/blob/main/modules/LoRA.py"

    %% Styles
    classDef frontend fill:#3498db,stroke:#2980b9,color:white
    classDef core fill:#2ecc71,stroke:#27ae60,color:white
    classDef loader fill:#f1c40f,stroke:#f39c12,color:black
    classDef ext fill:#9b59b6,stroke:#8e44ad,color:white
    classDef storage fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef config fill:#e74c3c,stroke:#c0392b,color:white
    classDef training fill:#1abc9c,stroke:#16a085,color:white
```
