# laitifranz/prompt2guard architecture

> Architecture diagram of the GitHub repository laitifranz/prompt2guard, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/laitifranz/prompt2guard
- Repository: https://github.com/laitifranz/prompt2guard

## Overview

##### Main Components
- Data & Configuration:
- Datasets and Data Preparation (see the README’s “Preparing Dataset” section).
- Configuration files in the “configs” directory (e.g., cddb_training.json and cddb_inference.json) that define experiment parameters.
- Model Architecture:
- The backbone VLM (CLIP) found in the “src/models/clip” folder (includes clip.py, model.py, prompt_learner.py, and a simple tokenizer).
- Additional model components like “src/models/slinet.py” for any supplementary modules.
- Methodology / Prompt Learning:
- The “src/methods” folder that contains prompt2guard.py (where the core method is implemented and the text-prompt conditioning is handled).
- Training & Evaluation Pipeline:
- Entry points: “src/train.py” for training and “src/eval.py” for evaluation.
- Supporting modules such as “src/trainer.py” and “src/utils” for data management, learning rate scheduling, zero-shot classification, and other utilities.
- Checkpoints and Pretrained Weights:
- The “checkpoints” directory holds pretrained weights, and this integrates into both training and evaluation pipelines.

##### Relationships and Data Flow
- Input Data Flow:
- Raw images are provided from external datasets (e.g., CDDB) and managed via “src/utils/data.py” and “src/utils/data_manager.py”.
- Classes and associated probabilities are loaded from “src/utils/classes.pkl”, produced through or used by “src/utils/zeroshot_classprediction.py”.
- Core Model Pipeline:
- Processed inputs are forwarded through the CLIP backbone (from the “src/models/clip” module).
Input Images -> Data Preparation -> CLIP backbone (with tokenization and prompt learning) -> Prompt2Guard module.
- The ensembling (combining predictions at various stages, as mentioned in the README configs) is applied after cosine similarity computation.
- Training and Monitoring:
- “src/train.py” orchestrates the training loop, calling on “src/trainer.py” and leveraging utilities for scheduling and checkpointing.
- Integration with wandb (Weights & Biases) for tracking metrics.
- Evaluation Pipeline:
- “src/eval.py” leverages the same model architecture with applied checkpoints to evaluate on multiple scenarios (cddb_hard and/or OOD datasets).

##### Architectural Patterns and Design Principles to Highlight
- Modularity: Separation of concerns is clear – models, methods, utilities, and configuration are isolated into separate directories.
- Config-driven Behavior: The use of JSON config files for training and inference enables easy reproducing and switching between scenarios.
- Extensibility: The clear separation of “prompt2guard” logic, coupled with the modular design of data handling and model definitions, supports future extensions or changes (e.g., using a different prompt optimizer).
- Efficiency: The design emphasizes prompt-conditioning to reduce computation (by avoiding multiple forward passes), which is a critical design principle in the method.

##### Final Advice for the Principal Software Engineer
- Don’t hesitate to break down complex nodes (like the model layer) into sub-components (e.g., token embedding, cosine similarity, ensembling logic) to capture the method’s multi-stage architecture.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Sources
    extData("External Deepfake Datasets"):::external
    classes("Classes (src/utils/classes.pkl)"):::data

    %% Data & Configuration Layer
    subgraph "Data & Configuration"
        DCT("configs/cddb_training.json"):::data
        DIE("configs/cddb_inference.json"):::data
        UDATA("src/utils/data.py"):::data
        UDM("src/utils/data_manager.py"):::data
        UZCP("src/utils/zeroshot_classprediction.py"):::data
    end

    %% Model Architecture Layer
    subgraph "Model Architecture"
        CLIP("CLIP Backbone & Prompt Learner (src/models/clip)"):::model
        SLINET("Additional Module: Slinet (src/models/slinet.py)"):::model
    end

    %% Methodology / Prompt Learning Layer
    subgraph "Methodology / Prompt Learning"
        PROMPT("Prompt Conditioning (src/methods/prompt2guard.py)"):::model
    end

    %% Training & Evaluation Pipeline Layer
    subgraph "Training & Evaluation Pipeline"
        TRAIN("Training Entry: train.py (src/train.py)"):::control
        TRAINER("Trainer (src/trainer.py)"):::control
        EVAL("Evaluation Entry: eval.py (src/eval.py)"):::control
        CHECK("Pretrained Weights (checkpoints/)"):::control
        WANDB("WandB Monitoring"):::control
    end

    %% Data Flow
    extData -->|"data_input"| UDATA
    UDATA -->|"preprocess"| UDM
    classes -->|"load_labels"| UZCP
    UDM -->|"manage_data"| CLIP
    UZCP -->|"class_prediction"| CLIP

    %% Model Flow
    CLIP -->|"forward_pass"| PROMPT
    CLIP -->|"ensemble_features"| SLINET

    %% Training & Evaluation Flow
    DCT -->|"load_config"| TRAIN
    DIE -->|"load_config"| TRAIN
    PROMPT -->|"conditioned_output"| TRAIN
    TRAIN -->|"orchestrates"| TRAINER
    TRAINER -->|"save_checkpoints"| CHECK
    CHECK -->|"load_weights"| EVAL
    EVAL -->|"metrics_log"| WANDB

    %% Click Events
    click DCT "https://github.com/laitifranz/prompt2guard/blob/main/configs/cddb_training.json"
    click DIE "https://github.com/laitifranz/prompt2guard/blob/main/configs/cddb_inference.json"
    click UDATA "https://github.com/laitifranz/prompt2guard/blob/main/src/utils/data.py"
    click UDM "https://github.com/laitifranz/prompt2guard/blob/main/src/utils/data_manager.py"
    click classes "https://github.com/laitifranz/prompt2guard/blob/main/src/utils/classes.pkl"
    click UZCP "https://github.com/laitifranz/prompt2guard/blob/main/src/utils/zeroshot_classprediction.py"
    click CLIP "https://github.com/laitifranz/prompt2guard/tree/main/src/models/clip"
    click SLINET "https://github.com/laitifranz/prompt2guard/blob/main/src/models/slinet.py"
    click PROMPT "https://github.com/laitifranz/prompt2guard/blob/main/src/methods/prompt2guard.py"
    click TRAIN "https://github.com/laitifranz/prompt2guard/blob/main/src/train.py"
    click TRAINER "https://github.com/laitifranz/prompt2guard/blob/main/src/trainer.py"
    click EVAL "https://github.com/laitifranz/prompt2guard/blob/main/src/eval.py"
    click CHECK "https://github.com/laitifranz/prompt2guard/tree/main/checkpoints/"

    %% Styles
    classDef external fill:#FFA07A,stroke:#000,stroke-width:2px;
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef model fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef control fill:#FFD700,stroke:#000,stroke-width:2px;
```
