# river-zhang/icedit architecture

> Architecture diagram of the GitHub repository river-zhang/icedit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/river-zhang/icedit
- Repository: https://github.com/river-zhang/icedit

## Overview

##### Project Type & Purpose
- This is an open‐source research codebase for “In-Context Edit,” a deep-learning image editing system built on a large diffusion transformer (Flux) with LoRA adapters.
- It covers two major subsystems:
a. Training pipeline (under `train/`) – data preparation, model implementation (Flux + LoRA), training loops, config management, logging.
b. Inference/Serving pipeline (under `scripts/`) – CLI inference, Gradio demo UI, model loading/offloading.

##### File Structure & Architectural Patterns
- Top‐level:
- `scripts/` – inference (`inference.py`), demo (`gradio_demo.py`), static config (`config.json`), `assets/` sample images.
- `train/` – training code, data (`parquet/`), training scripts & configs, source code split into Flux core and training modules.
- `requirements.txt` + `train/requirements.txt` – separate dependency sets for inference vs. training.
- Patterns:
- Pipeline pattern for both training and inference.
- Config‐driven workflows: YAML/JSON configs controlling hyperparameters, model paths, CPU/GPU offload flags.
- Modular layering:
– Core model library (`train/src/flux/…`)
– Training logic (`train/src/train/…`)
– CLI/API entrypoints (`scripts/…`)

##### Key Components
a. Data Preparation & Storage
- Raw training data in `train/parquet/`
- Preparation script `train/parquet/prepare.sh`
- Artifact storage: `train/runs/…` (checkpoints, logs, config snapshots)
b. Model Training Subsystem
- Flux core modules (`train/src/flux/`) implement diffusion transformer blocks, pipeline tools, LoRA controller.
- Training modules (`train/src/train/`): data loaders, callbacks, model definition, training loop.
- Config & orchestration (`train/train/config/*.yaml`, `train/train/script/train.sh`).
c. Model Artifact Management
- Checkpoints & LoRA weights exported after training.
- Optionally uploaded to Hugging Face Hub (external service).
d. Inference & Serving Subsystem
- CLI inference script (`scripts/inference.py`), reads image + instruction, loads base Flux model + LoRA weights, runs diffusion, outputs edited image.
- Gradio UI (`scripts/gradio_demo.py`), provides web interface for live editing, supports CPU offload flags, custom weight paths, quantized GGUF models.
- Config file (`scripts/config.json`) for default model paths, ports.
e. External Services & Integrations
- Hugging Face model repository & Spaces (hosting demo)
- ComfyUI workflows (external Git repos/install via Git URL)

##### Relationships & Data Flows
- Training Flow:
[Raw Images + Edit Instructions] → Data Prep → Flux+LoRA Training → Checkpoints & LoRA Weights → Model Registry / Local FS
- Inference Flow:
[User Input (Image + Instruction)] → CLI/Gradio → Load Flux Base Model + LoRA Weights → Diffusion Transformer Inference → Edited Image Output
- Deployment Flow:
Local Gradio or Hugging Face Space serves the inference component.

##### Architectural Patterns & Design Principles
- Clear separation of concerns: data prep, model core, training loops, inference endpoints.
- Configurable pipelines: externalize parameters in YAML/JSON to enable reproducibility and environment‐specific overrides.
- Modular extensibility:
- New LoRA variants by dropping new checkpoint files.
- Extend inference frontends (ComfyUI, Gradio, CLI) without touching core model code.

##### Final Checklist
- [ ] Clear, self-contained labels for each module/directory.
- [ ] Callouts for key config files (e.g., `.yaml`, `config.json`).
- [ ] Annotation of major dependencies (PyTorch/Hugging Face, Parquet, Gradio).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Training Pipeline
    subgraph "Training Pipeline"
        subgraph "Data Preparation & Storage"
            direction TB
            RawData[(Raw Training Data)]:::storage
            PrepScript["Data Preparation Script"]:::train
        end
        subgraph "Flux Core Model Library"
            direction TB
            Block["block.py"]:::train
            Condition["condition.py"]:::train
            Generate["generate.py"]:::train
            LORAControl["lora_controller.py"]:::train
            PipelineTools["pipeline_tools.py"]:::train
            Transformer["transformer.py"]:::train
        end
        subgraph "Training Logic & Pipeline"
            direction TB
            DataLoader["data.py"]:::train
            Callbacks["callbacks.py"]:::train
            ModelDef["model.py"]:::train
            TrainLoop["train.py"]:::train
        end
        subgraph "Training Config & Orchestration"
            direction TB
            TrainConfig["normal_lora.yaml"]:::config
            TrainScript["train.sh"]:::train
            TrainReadme["README.md"]:::config
            TrainReqs["requirements.txt"]:::config
        end
        subgraph "Artifact Storage"
            direction TB
            Runs[(Checkpoints & Logs)]:::storage
        end
    end

    %% Inference & Serving Pipeline
    subgraph "Inference & Serving"
        direction TB
        CLI["CLI Inference"]:::infer
        Gradio["Gradio Demo UI"]:::infer
        InfConfig["config.json"]:::config
        InferenceEngine["Inference Engine\n(Flux + LoRA)"]:::infer
        EditedImage["Edited Image Output"]:::infer
    end

    %% External Services
    subgraph "External Services"
        direction TB
        HFHub["Hugging Face Hub"]:::external
        Spaces["Hugging Face Space"]:::external
        ComfyUI["ComfyUI Node Registry"]:::external
    end

    %% Data Flows - Training
    RawData -->|"Parquet → DataLoader"| PrepScript
    PrepScript --> Block
    Block --> DataLoader
    Condition --> DataLoader
    LORAControl --> TrainLoop
    PipelineTools --> TrainLoop
    Transformer --> TrainLoop
    DataLoader --> TrainLoop
    Callbacks --> TrainLoop
    ModelDef --> TrainLoop
    TrainConfig --> TrainLoop
    TrainScript --> TrainLoop
    TrainLoop --> Runs
    Runs --> HFHub

    %% Data Flows - Inference
    CLI -->|"invokes"| InferenceEngine
    Gradio -->|"invokes"| InferenceEngine
    InfConfig --> InferenceEngine
    Runs -->|"loads LoRA Weights"| InferenceEngine
    HFHub -->|"fetches model"| InferenceEngine
    InferenceEngine -->|"produces"| EditedImage
    EditedImage --> Spaces
    InferenceEngine -->|"export workflow"| ComfyUI

    %% Click Events
    click RawData "https://github.com/river-zhang/icedit/tree/main/train/parquet/"
    click PrepScript "https://github.com/river-zhang/icedit/blob/main/train/parquet/prepare.sh"
    click Runs "https://github.com/river-zhang/icedit/tree/main/train/runs/"
    click Block "https://github.com/river-zhang/icedit/blob/main/train/src/flux/block.py"
    click Condition "https://github.com/river-zhang/icedit/blob/main/train/src/flux/condition.py"
    click Generate "https://github.com/river-zhang/icedit/blob/main/train/src/flux/generate.py"
    click LORAControl "https://github.com/river-zhang/icedit/blob/main/train/src/flux/lora_controller.py"
    click PipelineTools "https://github.com/river-zhang/icedit/blob/main/train/src/flux/pipeline_tools.py"
    click Transformer "https://github.com/river-zhang/icedit/blob/main/train/src/flux/transformer.py"
    click DataLoader "https://github.com/river-zhang/icedit/blob/main/train/src/train/data.py"
    click Callbacks "https://github.com/river-zhang/icedit/blob/main/train/src/train/callbacks.py"
    click ModelDef "https://github.com/river-zhang/icedit/blob/main/train/src/train/model.py"
    click TrainLoop "https://github.com/river-zhang/icedit/blob/main/train/src/train/train.py"
    click TrainConfig "https://github.com/river-zhang/icedit/blob/main/train/train/config/normal_lora.yaml"
    click TrainScript "https://github.com/river-zhang/icedit/blob/main/train/train/script/train.sh"
    click TrainReadme "https://github.com/river-zhang/icedit/blob/main/train/README.md"
    click TrainReqs "https://github.com/river-zhang/icedit/blob/main/train/requirements.txt"
    click CLI "https://github.com/river-zhang/icedit/blob/main/scripts/inference.py"
    click Gradio "https://github.com/river-zhang/icedit/blob/main/scripts/gradio_demo.py"
    click InfConfig "https://github.com/river-zhang/icedit/blob/main/scripts/config.json"

    %% Styles
    classDef train fill:#C6E5FF,stroke:#333,stroke-width:1px
    classDef infer fill:#C6F6D7,stroke:#333,stroke-width:1px
    classDef storage fill:#FFD8A6,stroke:#333,stroke-width:1px
    classDef config fill:#F5F5F5,stroke:#333,stroke-width:1px
    classDef external fill:#E0E0E0,stroke:#333,stroke-width:1px
```
