# haoyusimon/vl_debiasing architecture

> Architecture diagram of the GitHub repository haoyusimon/vl_debiasing, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haoyusimon/vl_debiasing
- Repository: https://github.com/haoyusimon/vl_debiasing

## Overview

##### Project type and purpose
- This is a machine-learning research codebase implementing “Joint Vision-Language Social Bias Removal for CLIP.”
- It provides scripts for training a debiased CLIP model over face-attribute datasets, measuring bias, and evaluating both bias reduction and vision-language performance.

##### File structure
Top-level files & dirs:
- clip_debiasing/
– datasets.py        ← data ingestion & preprocessing for FairFace, UTKFace, FACET
– measure_bias.py    ← routines to quantify social bias metrics
– models/            ← implementations of baseline CLIP and various debiasing strategies
– __init__.py
- train.py              ← orchestrates the training loop, optimization, checkpointing
- eval_all.py           ← end-to-end evaluation (bias & retrieval/classification metrics)
- eval_retrieval.py     ← specialized retrieval evaluation
- eval_train.py         ← evaluation on training set (sanity checks)
- utils.py              ← shared helpers (logging, config parsing, seeding)
- requirements.txt      ← exact PyTorch & Python dependencies

##### README for additional insights
- Confirms usage flow: install deps ➔ download face datasets ➔ run train.py ➔ run eval_all.py
- Notes external inspirations: ALBEF, mixed_KSG CLIP-clip, debias_vl — these are external model/code dependencies
- Outputs of training: exp/{version}/stdout.log, stderr.log, best.pth

##### Final advice
– Be exhaustively detailed: include each model file in the models folder as a subcomponent if needed
– Capture both control-flow (which script calls which modules) and data-flow (what data moves where)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "clip_debiasing"
        direction TB
        DL["Data Loader & Preprocessor"]:::internal
        MB["Bias Measurement Module"]:::internal
        subgraph "Model Zoo"
            direction TB
            MC["Baseline CLIP"]:::internal
            MPA["Prompt Debiasing (Age)"]:::internal
            MPG["Prompt Debiasing (Gender)"]:::internal
            MPR["Prompt Debiasing (Race)"]:::internal
            MVL["Joint V-L Debiaser"]:::internal
            MVL14["Joint V-L Debiaser (H14)"]:::internal
        end
    end

    FS["FairFace Dataset"]:::external
    UT["UTKFace Dataset"]:::external
    FA["FACET Dataset"]:::external

    PT["PyTorch"]:::external
    TV["torchvision"]:::external
    CL["Pretrained CLIP"]:::external

    UTILS["Shared Utilities"]:::internal

    TR["Training Script (train.py)"]:::internal
    EALL["End-to-End Evaluation (eval_all.py)"]:::internal
    ERET["Retrieval Eval (eval_retrieval.py)"]:::internal
    ETRN["Train Eval (eval_train.py)"]:::internal

    STORAGE["Experiment Store (exp/)"]:::storage

    FS --> DL
    UT --> DL
    FA --> DL

    DL --> TR
    DL --> ERET
    DL --> ETRN

    TR --> MC
    TR --> MPA
    TR --> MPG
    TR --> MPR
    TR --> MVL
    TR --> MVL14
    TR --> UTILS
    TR --> PT
    TR --> TV
    TR --> CL
    TR --> STORAGE

    STORAGE --> EALL
    STORAGE --> ERET
    STORAGE --> ETRN

    EALL --> MB
    EALL --> UTILS
    EALL --> PT
    EALL --> TV
    EALL --> CL

    ERET --> MB
    ERET --> UTILS

    ETRN --> UTILS

    click DL "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/datasets.py"
    click MB "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/measure_bias.py"
    click MC "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/models/model_clipped.py"
    click MPA "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/models/model_prompt_age.py"
    click MPG "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/models/model_prompt_gender.py"
    click MPR "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/models/model_prompt_race.py"
    click MVL "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/models/model_vl_debiasing.py"
    click MVL14 "https://github.com/haoyusimon/vl_debiasing/blob/main/clip_debiasing/models/model_vl_debiasing_h_14.py"
    click TR "https://github.com/haoyusimon/vl_debiasing/blob/main/train.py"
    click EALL "https://github.com/haoyusimon/vl_debiasing/blob/main/eval_all.py"
    click ERET "https://github.com/haoyusimon/vl_debiasing/blob/main/eval_retrieval.py"
    click ETRN "https://github.com/haoyusimon/vl_debiasing/blob/main/eval_train.py"
    click UTILS "https://github.com/haoyusimon/vl_debiasing/blob/main/utils.py"

    classDef internal fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef external fill:#98FB98,stroke:#000,stroke-width:1px
    classDef storage fill:#FFA500,stroke:#000,stroke-width:1px
```
