# tothebeginning/pulid architecture

> Architecture diagram of the GitHub repository tothebeginning/pulid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tothebeginning/pulid
- Repository: https://github.com/tothebeginning/pulid

## Overview

##### Project Type and Purpose
- PuLID is an open-source Python framework for personalized image generation via contrastive alignment.
- It provides multiple inference “apps” (app.py, app_flux.py, app_v1_1.py), a core pipeline for SDXL-based and FLUX-based models, and supporting libraries (eva_clip, flux).
- Use cases: local Gradio demos, HuggingFace/Replicate hosting, downstream integration.

##### File Structure Analysis
Top-level:
- app.py / app_flux.py / app_v1_1.py – entry points for different model versions (SDXL v1, v1.1, FLUX).
- docs/ – usage and model-specific documentation.
- eva_clip/ – CLIP encoding, model configs, utilities for vision-text representation.
- flux/ – minimal reimplementation of HuggingFace’s “FLUX” diffusion framework (math, sampling, modules).
- pulid/ – core PuLID pipelines and components (attention processors, encoders, main pipeline orchestrators).
- models/ – placeholder for downstream or exported models.
- example_inputs/ – sample assets for demos.
- pyproject.toml, requirements*.txt – build and dependency configuration.

##### Insights from README
- Two main pipelines: PuLID-v1/v1.1 (SDXL base) and PuLID-FLUX (FLUX base).
- Local inference via Gradio (app scripts).
- Core steps: encode reference image, encode prompt, contrastive alignment, diffusion sampling.
- External dependencies: PyTorch ≥2.0, Flux (for fp8 support), HuggingFace model hub.

a. Main Components
1. Client Layer
- CLI/Gradio UIs (app.py, app_v1_1.py, app_flux.py)
2. Service Layer
- PuLID Pipeline Modules (pulid/pipeline.py, pipeline_v1_1.py, pipeline_flux.py)
3. Domain Layer
- Encoders & Attention (pulid/encoders*.py, pulid/attention_processor.py)
- EVA-CLIP Model (eva_clip/*) for vision-text embedding
- Diffusion Model Core (Flux modules under flux/)
4. Infrastructure Layer
- PyTorch / Flux runtime & GPU
- HuggingFace Model Hub (external storage of .bin/.safetensors)
5. Supporting
- Configuration (eva_clip/model_configs, hf_configs)
- Utilities (eva_clip/utils.py, pulid/utils.py, flux/util.py)

b. Relationships & Data Flows
1. User inputs prompt + reference image via Gradio UI.
2. UI forwards inputs to the appropriate pipeline service.
3. Pipeline service:
- Calls EVA-CLIP encoder to extract image/text embeddings.
- Applies contrastive alignment modules.
- Feeds aligned features into diffusion sampler (from flux/math, flux/sampling).
4. Sampler invokes neural network layers (flux/modules/*, pulid/encoders_transformer).
5. Generated image returned to UI.
6. Model weights loaded from local cache or HuggingFace hub as configured.

c. Architectural Patterns & Principles
- Layered architecture: clear separation between presentation (apps), orchestration (pipelines), and computational kernels (eva_clip, flux).
- Plugin-style model configs: JSON‐driven model parameters (eva_clip/model_configs).
- Dependency inversion: pipelines depend on interface modules (encoders, attention) rather than concrete model implementations.
- Extensibility points: new pipelines can be added under pulid/ by following existing interfaces; new models can be configured via model_configs.

d. Key Technologies
- Python 3.9+
- PyTorch ≥2.0 or Flux-FP8 (PyTorch 2.4.1+)
- HuggingFace Transformers & Model Hub
- Gradio (for local UI demos)

## Diagram (Mermaid)

```mermaid
graph TD
    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        UI1["app.py (Gradio UI)"]:::presentation
        UI2["app_v1_1.py (Gradio UI)"]:::presentation
        UI3["app_flux.py (Gradio UI)"]:::presentation
    end

    %% Service Layer
    subgraph "Service Layer"
        direction TB
        PipelineBase["pulid/pipeline.py (Orchestration)"]:::service
        PipelineV11["pulid/pipeline_v1_1.py"]:::service
        PipelineFlux["pulid/pipeline_flux.py"]:::service
        UtilsShared["pulid/utils.py"]:::service
    end

    %% Domain Layer
    subgraph "Domain Layer"
        direction TB
        Encoders["pulid/encoders.py"]:::domain
        EncodersTrans["pulid/encoders_transformer.py"]:::domain
        AttentionProc["pulid/attention_processor.py"]:::domain
    end

    %% Computational Layer
    subgraph "Computational Layer"
        direction TB
        subgraph "EVA-CLIP"
            direction TB
            EVACFactory["eva_clip/factory.py"]:::compute
            EVAModel["eva_clip/model.py"]:::compute
            EVAVit["eva_clip/eva_vit_model.py"]:::compute
            EVAHFModel["eva_clip/hf_model.py"]:::compute
            EVAHFConfig["eva_clip/hf_configs.py"]:::compute
            EVATokenizer["eva_clip/tokenizer.py"]:::compute
            EVAUtils["eva_clip/utils.py"]:::compute
            EVAConfigs["eva_clip/model_configs/*.json"]:::compute
        end
        subgraph "FLUX Diffusion"
            direction TB
            FLUXMath["flux/math.py"]:::compute
            FLUXSampling["flux/sampling.py"]:::compute
            FLUXModel["flux/model.py"]:::compute
            FLUXModules["flux/modules/*"]:::compute
            FLUXUtils["flux/util.py"]:::compute
        end
    end

    %% Infrastructure Layer
    subgraph "Infrastructure / External"
        direction TB
        HFHub[("HuggingFace Model Hub")]:::external
        GPUEnv(("GPU Runtime\n& Python Env")):::external
    end

    %% Connections
    UI1 -->|"invoke"| PipelineBase
    UI2 -->|"invoke"| PipelineBase
    UI3 -->|"invoke"| PipelineBase

    PipelineBase -->|"routes to v1.1"| PipelineV11
    PipelineBase -->|"routes to FLUX"| PipelineFlux
    PipelineBase -->|load weights| HFHub
    PipelineBase -->|shared utils| UtilsShared

    PipelineV11 -->|"embed/image/text"| Encoders
    PipelineV11 -->|"transformer"| EncodersTrans
    PipelineV11 -->|"attention"| AttentionProc
    PipelineV11 -->|"use EVA-CLIP"| EVACFactory
    PipelineV11 -->|"run on"| GPUEnv

    PipelineFlux -->|"embed/image/text"| Encoders
    PipelineFlux -->|"transformer"| EncodersTrans
    PipelineFlux -->|"attention"| AttentionProc
    PipelineFlux -->|"use EVA-CLIP"| EVACFactory
    PipelineFlux -->|"diffusion"| FLUXSampling
    PipelineFlux -->|"math ops"| FLUXMath
    PipelineFlux -->|"core model"| FLUXModel
    PipelineFlux -->|"modules"| FLUXModules
    PipelineFlux -->|"utils"| FLUXUtils
    PipelineFlux -->|"run on"| GPUEnv

    Encoders -->|"tokenize"| EVATokenizer
    Encoders -->|"config"| EVAConfigs
    Encoders -->|"init"| EVAVit
    EVACFactory --> EVAModel
    EVAModel --> EVAHFModel
    EVAHFModel --> EVAHFConfig
    EVAModel --> EVAUtils

    %% Click Events
    click UI1 "https://github.com/tothebeginning/pulid/blob/main/app.py"
    click UI2 "https://github.com/tothebeginning/pulid/blob/main/app_v1_1.py"
    click UI3 "https://github.com/tothebeginning/pulid/blob/main/app_flux.py"
    click PipelineBase "https://github.com/tothebeginning/pulid/blob/main/pulid/pipeline.py"
    click PipelineV11 "https://github.com/tothebeginning/pulid/blob/main/pulid/pipeline_v1_1.py"
    click PipelineFlux "https://github.com/tothebeginning/pulid/blob/main/pulid/pipeline_flux.py"
    click UtilsShared "https://github.com/tothebeginning/pulid/blob/main/pulid/utils.py"
    click Encoders "https://github.com/tothebeginning/pulid/blob/main/pulid/encoders.py"
    click EncodersTrans "https://github.com/tothebeginning/pulid/blob/main/pulid/encoders_transformer.py"
    click AttentionProc "https://github.com/tothebeginning/pulid/blob/main/pulid/attention_processor.py"
    click EVACFactory "https://github.com/tothebeginning/pulid/blob/main/eva_clip/factory.py"
    click EVAModel "https://github.com/tothebeginning/pulid/blob/main/eva_clip/model.py"
    click EVAVit "https://github.com/tothebeginning/pulid/blob/main/eva_clip/eva_vit_model.py"
    click EVAHFModel "https://github.com/tothebeginning/pulid/blob/main/eva_clip/hf_model.py"
    click EVAHFConfig "https://github.com/tothebeginning/pulid/blob/main/eva_clip/hf_configs.py"
    click EVATokenizer "https://github.com/tothebeginning/pulid/blob/main/eva_clip/tokenizer.py"
    click EVAUtils "https://github.com/tothebeginning/pulid/blob/main/eva_clip/utils.py"
    click EVAConfigs "https://github.com/tothebeginning/pulid/tree/main/eva_clip/model_configs"
    click FLUXMath "https://github.com/tothebeginning/pulid/blob/main/flux/math.py"
    click FLUXSampling "https://github.com/tothebeginning/pulid/blob/main/flux/sampling.py"
    click FLUXModel "https://github.com/tothebeginning/pulid/blob/main/flux/model.py"
    click FLUXModules "https://github.com/tothebeginning/pulid/tree/main/flux/modules"
    click FLUXUtils "https://github.com/tothebeginning/pulid/blob/main/flux/util.py"

    %% Styles
    classDef presentation fill:#D6E9F8,stroke:#1F77B4,color:#1F77B4
    classDef service fill:#DFF0D8,stroke:#28A745,color:#28A745
    classDef domain fill:#FCF8E3,stroke:#F0AD4E,color:#F0AD4E
    classDef compute fill:#FDE9D9,stroke:#E67E22,color:#E67E22
    classDef external fill:#EEEEEE,stroke:#999999,color:#555555
```
