# hidream-ai/hidream-e1 architecture

> Architecture diagram of the GitHub repository hidream-ai/hidream-e1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hidream-ai/hidream-e1
- Repository: https://github.com/hidream-ai/hidream-e1

## Overview

##### Project Type and Purpose
- This is an open-source, instruction-based image editing tool built on top of a pretrained diffusion backbone (HiDream-I1). It provides CLI scripts (`inference.py`, `instruction_refinement.py`) and a Gradio demo (`gradio_demo.py`) to let end users and developers perform and refine image edits via natural-language instructions.

##### File-Structure Analysis
- Top level:
- `inference.py` – main CLI entry point for image editing inference.
- `instruction_refinement.py` – CLI for improving user instructions via a VLM or OpenAI API.
- `pipeline_hidream_image_editing.py` – defines the custom Diffusers pipeline that glues together tokenizer, text encoder, image transformer, and LoRA adapters.
- `gradio_demo.py` – launches an interactive web UI.
- `assets/` – sample images and demo visuals.
- `requirements.txt` – Python dependencies.
- No explicit frontend/backend split—this is a library/tool with two user-facing entrypoints (CLI and web demo) and a core model pipeline.

##### Key Components
a. User Interfaces
- CLI Inference (`inference.py`)
- Instruction Refinement CLI (`instruction_refinement.py`)
- Gradio Web UI (`gradio_demo.py`)
b. Core Pipeline
- Instruction Refinement Module (optional; calls VLM or OpenAI API)
- HiDream Image Editing Pipeline (inherits Diffusers Pipeline):
- Diffusion backbone model (HiDream-I1 transformer)
- LoRA adapter layers (injected via `peft`)
- Image preprocessor / postprocessor (PIL-based resize, normalization)
c. External Services & Repositories
- HuggingFace Hub (model checkpoints for Llama and HiDream)
- VLM API or local vLLM (for instruction refinement)
d. Data Flow
1. User submits: source image + raw instruction
2. (Optional) Instruction Refinement Module returns refined instruction
3. Tokenizer & text encoder encode prompt
4. Pipeline drives diffusion steps: editing stage + optional refinement stage
5. Pipeline outputs an edited image tensor → PIL image → saved or returned to UI

##### Architectural Patterns & Design Principles
- Modular Pipeline Pattern: each stage (tokenization, text encoding, diffusion editing, refinement) is encapsulated.
- Adapter-based extensibility: LoRA adapters let you fine-tune or swap in new editing behaviors without retraining entire model.
- Dual-mode UI: reuse the same core pipeline for CLI and web demo, separating presentation from business logic.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Interfaces"
        CLIInf["CLI Inference Interface\n(inference.py)"]:::ui
        IRCli["Instruction Refinement CLI\n(instruction_refinement.py)"]:::ui
        WebUI["Gradio Web UI\n(gradio_demo.py)"]:::ui
    end

    subgraph "Core Pipeline"
        Token["Tokenizer & Text Encoder"]:::core
        InstRefMod["Instruction Refinement Module"]:::core
        Pipeline["HiDreamImageEditingPipeline\n(pipeline_hidream_image_editing.py)"]:::core
    end

    subgraph "External Services"
        HFHub["HuggingFace Hub"]:::external
        VLM["VLM API / vLLM"]:::external
    end

    subgraph "Project Files"
        Assets["Sample Images & Demo Visuals\n(assets/)"]:::doc
        Req["Dependency Specification\n(requirements.txt)"]:::doc
        Readme["Project Documentation\n(README.md)"]:::doc
        Lic["License Information\n(LICENSE)"]:::doc
        Gitignore["Git Ignore Rules\n(.gitignore)"]:::doc
    end

    CLIInf -->|"source image + raw_instruction"| InstRefMod
    InstRefMod -->|"refined_instruction"| Token
    Token -->|"encoded_prompt"| Pipeline
    Pipeline -->|"edited_image_tensor → PIL image"| CLIInf
    Pipeline -->|"edited_image_tensor → PIL image"| WebUI

    CLIInf -->|"uses pipeline"| Pipeline
    WebUI -->|"uses pipeline"| Pipeline

    InstRefMod -->|"calls"| VLM
    IRCli -->|"raw_instruction"| VLM
    VLM -->|"refined_instruction"| IRCli

    HFHub -->|"model & LoRA weights download"| Pipeline

    classDef ui fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef core fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px
    classDef doc fill:#D3D3D3,stroke:#333,stroke-width:1px

    click CLIInf "https://github.com/hidream-ai/hidream-e1/blob/main/inference.py"
    click IRCli "https://github.com/hidream-ai/hidream-e1/blob/main/instruction_refinement.py"
    click WebUI "https://github.com/hidream-ai/hidream-e1/blob/main/gradio_demo.py"
    click Pipeline "https://github.com/hidream-ai/hidream-e1/blob/main/pipeline_hidream_image_editing.py"
    click Assets "https://github.com/hidream-ai/hidream-e1/tree/main/assets/"
    click Req "https://github.com/hidream-ai/hidream-e1/blob/main/requirements.txt"
    click Readme "https://github.com/hidream-ai/hidream-e1/blob/main/README.md"
    click Lic "https://github.com/hidream-ai/hidream-e1/tree/main/LICENSE"
    click Gitignore "https://github.com/hidream-ai/hidream-e1/blob/main/.gitignore"
```
