# zqhang/anomalyclip architecture

> Architecture diagram of the GitHub repository zqhang/anomalyclip, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zqhang/anomalyclip
- Repository: https://github.com/zqhang/anomalyclip

## Overview

##### Project type and purpose
- This is an open-source research codebase for zero-shot anomaly detection, named AnomalyCLIP.
- Purpose: Adapt a pre-trained vision-language model (CLIP) with object-agnostic text prompts to detect and segment anomalies across diverse industrial and medical imaging datasets without requiring target-domain training samples.

##### File structure
Top‐level layout:
- AnomalyCLIP_lib/            ← Core library implementing model building, CLIP wrapper, tokenizers, transforms, prompt learning
- generate_dataset_json/      ← Dataset-specific JSON generators for all supported datasets
- train.py, test.py, test_one_example.py          ← High-level scripts orchestrating training and inference
- train.sh, test.sh, test_one_example.sh          ← Shell wrappers for reproducibility
- dataset.py                   ← Registry of dataset metadata
- loss.py, metrics.py          ← Custom loss and evaluation metrics
- prompt_ensemble.py           ← Ensemble logic for multiple prompt templates
- logger.py                    ← Logging utility
- visualization.py             ← Utility to render anomaly maps and plots
- utils.py                     ← Miscellaneous helper functions
- checkpoints/, results/       ← Model checkpoints and experiment outputs
- assets/overview.pdf          ← High-level architecture overview

Architectural patterns:
- Modular separation of data preparation, core model library, training pipeline, evaluation & visualization.
- Script-driven orchestration (train/test), library functions for model internals.

##### Key insights from README
- Framework: PyTorch 2.0.0
- Relies on an external open_clip or built-in CLIP implementation (AnomalyCLIP_lib/CLIP.py).
- Workflow:
a. Download raw images →
b. Run generate_dataset_json to produce a unified JSON manifest →
c. train.py uses manifest + AnomalyCLIP_lib to build and train model once on MVTec AD →
d. test.py zero-shot tests on all other datasets →
e. visualization and logging produce output artifact.

##### Final checklist
- Explicit callouts for key files/scripts (train.py, test.sh, generate_dataset_json/*)
- Notes on external dependencies (PyTorch, CLIP, tokenizers)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        direction TB
        RawDatasets["Raw Datasets (MVTec, VisA, MPDD, …)"]:::data
        GenJSON["generate_dataset_json/"]:::data
        JSONManifest["JSON Manifests"]:::data
        DatasetRegistry["dataset.py"]:::data
    end

    %% Core Library
    subgraph "Core Library (AnomalyCLIP_lib)"
        direction TB
        ACLIPInit["__init__.py"]:::core
        ACLIPMain["AnomalyCLIP.py"]:::core
        CLIPWrapper["CLIP.py"]:::core
        ModelBuilder["build_model.py"]:::core
        ModelLoader["model_load.py"]:::core
        Tokenizer["simple_tokenizer.py"]:::core
        Vocab["bpe_simple_vocab_16e6.txt.gz"]:::core
        Transforms["transform.py"]:::core
        Constants["constants.py"]:::core
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        TrainScript["train.py"]:::script
        TrainShell["train.sh"]:::script
        DataLoader["DataLoader (reads JSON)"]:::script
        ForwardPass["Forward Pass (CLIP vision + prompts)"]:::script
        LossComp["loss.py"]:::script
        PromptEns["prompt_ensemble.py"]:::script
        LoggerTrain["logger.py"]:::script
        Utils["utils.py"]:::script
        CheckpointsDir["checkpoints/"]:::output
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        direction TB
        TestBatch["test.py"]:::script
        TestOne["test_one_example.py"]:::script
        TestShell1["test.sh"]:::script
        TestShell2["test_one_example.sh"]:::script
        MetricsComp["metrics.py"]:::script
        ModelLoadInf["model_load.py"]:::core
        CheckpointsUse["checkpoints/"]:::output
        JSONUse["JSON Manifests"]:::data
        ResultsDir["results/"]:::output
    end

    %% Visualization & Logging
    subgraph "Visualization & Logging"
        direction TB
        Visualization["visualization.py"]:::output
        LoggerAll["logger.py"]:::script
        ResultsRead["results/"]:::output
        CheckpointsRead["checkpoints/"]:::output
    end

    %% Experiment Artifacts
    subgraph "Experiment Artifacts"
        direction TB
        ArtifactsCheck["checkpoints/"]:::output
        ArtifactsResults["results/"]:::output
    end

    %% Connections
    RawDatasets -->|generate_dataset_json| GenJSON
    GenJSON -->|produces| JSONManifest
    JSONManifest -->|loaded by| DataLoader
    DatasetRegistry -->|provides metadata| DataLoader

    TrainScript -->|calls| DataLoader
    TrainScript -->|calls| ACLIPMain
    TrainScript -->|calls| CLIPWrapper
    TrainScript -->|calls| ModelBuilder
    TrainScript -->|calls| ModelLoader
    TrainScript -->|calls| Tokenizer
    TrainScript -->|calls| Transforms
    TrainScript -->|uses| Constants
    TrainScript -->|uses| Utils
    TrainScript -->|calls| LossComp
    TrainScript -->|calls| PromptEns
    TrainScript -->|logs to| LoggerTrain
    TrainScript -->|writes| CheckpointsDir

    TestBatch -->|reads| CheckpointsUse
    TestBatch -->|reads| JSONUse
    TestBatch -->|calls| ACLIPMain
    TestBatch -->|calls| ModelLoadInf
    TestBatch -->|calls| MetricsComp
    TestBatch -->|writes| ResultsDir

    TestOne -->|reads| CheckpointsUse
    TestOne -->|reads| JSONUse
    TestOne -->|calls| ACLIPMain
    TestOne -->|calls| ModelLoadInf
    TestOne -->|calls| MetricsComp
    TestOne -->|writes| ResultsDir

    LoggerAll -->|records| ArtifactsResults
    Visualization -->|reads| ResultsRead
    Visualization -->|reads| CheckpointsRead
    Visualization -->|outputs| ArtifactsResults

    %% Artifacts grouping
    CheckpointsDir ---|alias| ArtifactsCheck
    ResultsDir ---|alias| ArtifactsResults

    %% Click Events
    click GenJSON "https://github.com/zqhang/anomalyclip/tree/main/generate_dataset_json/"
    click DatasetRegistry "https://github.com/zqhang/anomalyclip/blob/main/dataset.py"
    click ACLIPInit "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/__init__.py"
    click ACLIPMain "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/AnomalyCLIP.py"
    click CLIPWrapper "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/CLIP.py"
    click ModelBuilder "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/build_model.py"
    click ModelLoader "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/model_load.py"
    click Tokenizer "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/simple_tokenizer.py"
    click Vocab "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/bpe_simple_vocab_16e6.txt.gz"
    click Transforms "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/transform.py"
    click Constants "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/constants.py"
    click TrainScript "https://github.com/zqhang/anomalyclip/blob/main/train.py"
    click TrainShell "https://github.com/zqhang/anomalyclip/blob/main/train.sh"
    click LossComp "https://github.com/zqhang/anomalyclip/blob/main/loss.py"
    click PromptEns "https://github.com/zqhang/anomalyclip/blob/main/prompt_ensemble.py"
    click LoggerTrain "https://github.com/zqhang/anomalyclip/blob/main/logger.py"
    click Utils "https://github.com/zqhang/anomalyclip/blob/main/utils.py"
    click CheckpointsDir "https://github.com/zqhang/anomalyclip/tree/main/checkpoints/"
    click TestBatch "https://github.com/zqhang/anomalyclip/blob/main/test.py"
    click TestOne "https://github.com/zqhang/anomalyclip/blob/main/test_one_example.py"
    click TestShell1 "https://github.com/zqhang/anomalyclip/blob/main/test.sh"
    click TestShell2 "https://github.com/zqhang/anomalyclip/blob/main/test_one_example.sh"
    click MetricsComp "https://github.com/zqhang/anomalyclip/blob/main/metrics.py"
    click ModelLoadInf "https://github.com/zqhang/anomalyclip/blob/main/AnomalyCLIP_lib/model_load.py"
    click Visualization "https://github.com/zqhang/anomalyclip/blob/main/visualization.py"
    click ResultsDir "https://github.com/zqhang/anomalyclip/tree/main/results/"

    %% Styles
    classDef data fill:#D0E4FE,stroke:#0366D6,color:#0366D6
    classDef core fill:#D6F5D6,stroke:#228B22,color:#228B22
    classDef script fill:#FFE5B4,stroke:#FF8C00,color:#FF8C00
    classDef output fill:#E6D3F8,stroke:#800080,color:#800080
```
