# joycenerd/p4d architecture

> Architecture diagram of the GitHub repository joycenerd/p4d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/joycenerd/p4d
- Repository: https://github.com/joycenerd/p4d

## Overview

##### Project type and purpose
- This is an open-source research tool (Prompting4Debugging, P4D) for red-teaming text-to-image diffusion models by finding problematic prompts.
- It provides two main optimization routines (P4D-N and P4D-K) to attack deployed safe diffusion pipelines (ESD, SLD, SD-NEGP).
- It is a CLI-driven Python project with data processing, prompt optimization, safety model pipelines, and classification modules.

##### File structure
Top-level scripts and CLI entry points
- run_p4dn.py, run_p4dk.py: orchestration for P4D-N and P4D-K runs
- process_data.py: preprocessing and formatting raw prompt datasets
Configuration and data files
- configs/*.json: model-specific hyperparameters and pipeline settings
- data/*.csv: original and processed prompt datasets
Core modules
- prompt_optimization/: optimization algorithms (optimize_n.py for P4D-N, optimize_k.py for P4D-K)
- model/: custom pipelines built on top of Stable Diffusion (modified_sld_pipeline.py, modified_stable_diffusion_pipeline.py) and UNet (unet.py)
- nudenet/: integrates NudeNet for NSFW classification and detection
- open_clip/: OpenAI CLIP implementation for prompt encoding, model configs, and utility functions
Utilities and global scripts
- main_utils.py: shared helper functions
- figure/: visualization assets for the README

##### Additional insights from the README
- Describes two optimization workflows: P4D-N (needle optimization) and P4D-K (key optimization)
- Highlights required components:
– Pretrained safe pipelines (ESD UNet checkpoints, SLD, SD-NEGP)
– NudeNet ONNX classifier model
– Torch and CUDA dependencies
- Emphasizes outputs: optimized prompt CSVs, generated images, logs

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI and Configuration
    subgraph "CLI Entrypoints & Config" 
        direction TB
        runN["run_p4dn.py"]:::cli
        runK["run_p4dk.py"]:::cli
        proc["process_data.py"]:::cli
        subgraph "Configuration Loader"
            direction TB
            cfg1["esd_nudity_optmz_config.json"]:::cli
            cfg2["sdneg_nudity_optmz_config.json"]:::cli
            cfg3["sldMAX_nudity_optmz_config.json"]:::cli
            cfg4["sldSTRONG_nudity_optmz_config.json"]:::cli
        end
    end

    %% Data Loading
    subgraph "Data Loader & Preprocessor"
        direction TB
        csv["Prompt CSVs"]:::data
    end

    %% Optimization Engine
    subgraph "Prompt Optimization Engine"
        direction TB
        optN["optimize_n.py"]:::opt
        optK["optimize_k.py"]:::opt
    end

    %% Optimization Loop Components
    subgraph "Optimization Loop"
        direction TB
        enc["OpenCLIP Encoder"]:::inf
        subgraph "Safe T2I Pipelines"
            direction TB
            p4dnSD["P4D-N SD Pipeline"]:::inf
            p4dnSLD["P4D-N SLD Pipeline"]:::inf
            p4dkSD["P4D-K SD Pipeline"]:::inf
            p4dkSLD["P4D-K SLD Pipeline"]:::inf
            unet["UNet Model"]:::inf
        end
        nn["NudeNet Classifier"]:::cls
        loss["Loss Computation"]:::opt
    end

    %% Output
    out["Output Manager"]:::data

    %% Connections
    runN -->|reads| cfg1
    runN -->|reads| cfg2
    runK -->|reads| cfg3
    runK -->|reads| cfg4
    runN --> proc
    runK --> proc
    proc --> csv
    csv --> optN
    csv --> optK

    optN --> enc
    optK --> enc

    enc --> p4dnSD
    enc --> p4dnSLD
    enc --> p4dkSD
    enc --> p4dkSLD
    unet --> p4dnSD
    unet --> p4dkSD
    unet --> p4dnSLD
    unet --> p4dkSLD

    p4dnSD --> nn
    p4dnSLD --> nn
    p4dkSD --> nn
    p4dkSLD --> nn

    nn --> loss
    loss --> optN
    loss --> optK

    optN --> out
    optK --> out

    %% Click Events
    click runN "https://github.com/joycenerd/p4d/blob/main/run_p4dn.py"
    click runK "https://github.com/joycenerd/p4d/blob/main/run_p4dk.py"
    click proc "https://github.com/joycenerd/p4d/blob/main/process_data.py"
    click cfg1 "https://github.com/joycenerd/p4d/blob/main/configs/esd_nudity_optmz_config.json"
    click cfg2 "https://github.com/joycenerd/p4d/blob/main/configs/sdneg_nudity_optmz_config.json"
    click cfg3 "https://github.com/joycenerd/p4d/blob/main/configs/sldMAX_nudity_optmz_config.json"
    click cfg4 "https://github.com/joycenerd/p4d/blob/main/configs/sldSTRONG_nudity_optmz_config.json"
    click optN "https://github.com/joycenerd/p4d/blob/main/prompt_optimization/optimize_n.py"
    click optK "https://github.com/joycenerd/p4d/blob/main/prompt_optimization/optimize_k.py"
    click p4dnSD "https://github.com/joycenerd/p4d/blob/main/model/p4dn/modified_stable_diffusion_pipeline.py"
    click p4dnSLD "https://github.com/joycenerd/p4d/blob/main/model/p4dn/modified_sld_pipeline.py"
    click p4dkSD "https://github.com/joycenerd/p4d/blob/main/model/p4dk/modified_stable_diffusion_pipeline.py"
    click p4dkSLD "https://github.com/joycenerd/p4d/blob/main/model/p4dk/modified_sld_pipeline.py"
    click unet "https://github.com/joycenerd/p4d/blob/main/model/unet.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/classifier.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/lite_classifier.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/detector.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/detector_utils.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/run_classify.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/classify_pil.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/image_utils.py"
    click nn "https://github.com/joycenerd/p4d/blob/main/nudenet/video_utils.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/__init__.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/factory.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/hf_model.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/model.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/tokenizer.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/transform.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/transformer.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/utils.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/pretrained.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/modified_resnet.py"
    click enc "https://github.com/joycenerd/p4d/blob/main/open_clip/openai.py"
    click enc "https://github.com/joycenerd/p4d/tree/main/open_clip/model_configs"

    %% Styles
    classDef data fill:#add8e6,stroke:#333,stroke-width:1px
    classDef opt fill:#90ee90,stroke:#333,stroke-width:1px
    classDef inf fill:#ffa500,stroke:#333,stroke-width:1px
    classDef cls fill:#ff7f7f,stroke:#333,stroke-width:1px
    classDef cli fill:#d3d3d3,stroke:#333,stroke-width:1px
```
