# yuanshi9815/ominicontrol architecture

> Architecture diagram of the GitHub repository yuanshi9815/ominicontrol, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yuanshi9815/ominicontrol
- Repository: https://github.com/yuanshi9815/ominicontrol

## Overview

##### Project Type and Purpose
- The project is a full-stack research project that includes both inference and training pipelines.
- Its primary goal is to provide a control framework for diffusion transformers with minimal modifications. The design supports subject-driven generation and spatial tasks.
- The README and file structure indicate components for running demonstrations (Gradio app), training (multiple scripts and configuration YAML files), and example notebooks for usage.

##### Analyzing the File Structure
- Top-Level Overview:
- “assets”: Contains images (e.g., demos used in the README).
- “examples”: Notebooks demonstrating different generation tasks (subject-driven, spatial, etc.).
- “src”: Contains primary source code:
- “flux”: Implements the core modules for control injection (files like block.py, condition.py, generate.py, lora_controller.py, pipeline_tools.py, transformer.py) that extend or interface with the FLUX diffusion transformer.
- “gradio”: Hosts the Gradio app (gradio_app.py) that serves as the frontend for subject-driven inference.
- “train”: Contains modules for training (data handling, model definitions, callback functions, and the main train script).
- “train” Directory (at the top level):
- Contains additional training documentation (a README), configuration YAML files for several tasks (canny, cartoon, etc.) and shell scripts for data download and initiating training.

##### Architectural Components
a. Core Model & Control Modules:
- Block, Condition, Generate, LoRA Controller, Pipeline Tools, Transformer.
b. Training Pipeline:
- Separate from the inference path, the training modules (src/train and train/) encompass data preprocessing, model training (callbacks, model architecture, training scripts), and configuration management. Represent these as a “Training Pipeline” block that receives configuration (via YAML files) and invokes training scripts (shell scripts under train/script).
c. Frontend Interface:
d. Example Notebooks and Assets:
e. External Integrations and Deployment:

##### Highlighting Relationships and Interactions
- From the Gradio UI to the backend “Flux” modules: indicate that user inputs (image and prompt) are fed into the diffusion transformer pipeline.
- Within the backend, show interaction among components (e.g., “Condition” feeds into “Transformer”, “LoRA Controller” modulates model parameters).
- Separate workflows:
- Inference workflow (Gradio interface → Flux modules) versus
- Training workflow (data -> training scripts → model updates).

##### Noting Key Architectural Patterns and Design Principles
- Modularity: The system is split into distinct modules – core diffusion control (flux), training, and user interaction (Gradio app).
- Extensibility: The organization of training scripts and configuration files supports easy adaptation for different tasks (subject, inpainting, cartoons, etc.).
- Separation of Concerns: The structure shows clear separations between training, inference, and demonstration components, making maintenance and extension easier.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface Layer
    subgraph "User Interface"
        UI["Gradio Web UI"]:::frontend
    end

    %% OminiControl Core Logic
    subgraph "OminiControl Logic"
        DT["Diffusion Transformer (FLUX)"]:::core
        subgraph "Core Modules"
            B["Block"]:::core
            C["Condition"]:::core
            TR["Transformer"]:::core
            G["Generate"]:::core
            L["LoRA Controller"]:::core
            P["Pipeline Tools"]:::core
        end
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        TS["Training Modules (src/train)"]:::training
        TC["Training Configs (train)"]:::training
    end

    %% External Documentation & Integrations
    subgraph "Documentation & External"
        EX["Examples (Jupyter Notebooks)"]:::external
        DA["Demo Assets"]:::external
        EG["HuggingFace/GitHub"]:::external
    end

    %% Data Flow Connections
    UI -->|"input"| DT
    DT -->|"includes"| B
    DT -->|"includes"| C
    DT -->|"includes"| P
    C -->|"processes"| TR
    L -->|"modulates"| TR
    TR -->|"produces"| G

    TS -->|"triggers_update"| DT
    TC -->|"provides_config"| TS

    %% External Connections (informational)
    EX ---|"documentation"| UI
    DA ---|"assets"| UI
    EG ---|"integration"| UI

    %% Click Events
    click UI "https://github.com/yuanshi9815/ominicontrol/blob/main/src/gradio/gradio_app.py"
    click DT "https://github.com/yuanshi9815/ominicontrol/tree/main/src/flux/"
    click TS "https://github.com/yuanshi9815/ominicontrol/tree/main/src/train/"
    click TC "https://github.com/yuanshi9815/ominicontrol/tree/main/train/"
    click EX "https://github.com/yuanshi9815/ominicontrol/tree/main/examples/"
    click DA "https://github.com/yuanshi9815/ominicontrol/tree/main/assets/demo/"

    %% Styles
    classDef frontend fill:#CCE5FF,stroke:#003399,stroke-width:2px;
    classDef core fill:#CCFFCC,stroke:#006600,stroke-width:2px;
    classDef training fill:#FFE6CC,stroke:#CC6600,stroke-width:2px;
    classDef external fill:#FFCCCC,stroke:#990000,stroke-width:2px;
```
