# zqhang/accurate-winclip-pytorch architecture

> Architecture diagram of the GitHub repository zqhang/accurate-winclip-pytorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zqhang/accurate-winclip-pytorch
- Repository: https://github.com/zqhang/accurate-winclip-pytorch

## Overview

##### Project type and purpose
- This is an open-source machine-learning toolkit (“WinCLIP” / “AnomalyCLIP”) for zero-shot and few-shot anomaly detection/segmentation built on top of the OpenAI-inspired CLIP model (via the open_clip codebase).
- It provides data ingestion from standard anomaly datasets (MVTec, VisA), wraps an open_clip CLIP backbone, implements WinCLIP anomaly scoring, and delivers evaluation scripts.

##### File structure
- Top-level scripts:
- zero_shot.sh, few_shot.sh, reproduce_WinCLIP.py
- test.py (smoke tests)
- Data layer:
- data/ (dataset adapters: mvtec.py, visa.py)
- dataset.py (unified dataset interface)
- Model layer:
- src/open_clip/ (CLIP implementation and wrappers, model configs, tokenizers, generation utilities)
- Training & inference:
- src/training/ – data loaders, distributed training, scheduler, precision, logger, main entrypoints (train.py, zero_shot.py, few_shot.py)
- Configuration & dependencies:
- requirements.txt
- bpe-vocabulary, JSON model configs
- External integration:
- push_to_hf_hub.py (Hugging Face Hub)

##### Core components
a. Data ingestion & preprocessing
b. Model backbone
- “open_clip backbone” (ViT/ResNet models, tokenizer, transform)
- “WinCLIP anomaly head” (coca_model.py, generation_utils.py, loss.py)
c. Training pipeline
- “Training orchestrator” (train.py) → modules: distributed.py, scheduler.py, precision.py, logger.py
d. Zero-Shot & Few-Shot inference pipelines
- “Zero-shot runner” (zero_shot.py + zero_shot.sh)
- “Few-shot runner” (few_shot.py + few_shot.sh)
e. External services
- “Hugging Face Hub” (push_to_hf_hub.py)

##### Relationships & data flows
- Data → Dataset adapter → Preprocessed tensor batches
- Batches → Model backbone → Feature embeddings
- Embeddings → Anomaly scoring head → Anomaly maps / labels
- Training orchestrator ↔ scheduler/precision/logger → updates Model backbone weights
- Inference runners call model in eval mode, write metrics to disk
- Optionally push trained weights to HF Hub

##### Architectural patterns & principles
- Modular separation: Data layer, Model layer, Training layer, Inference layer
- Config-driven: All backbones and hyperparameters are in JSON and params.py
- Extensibility: Add new datasets by dropping adapters in data/, add new backbones by supplying JSON in model_configs/
- Reproducibility: Shell wrappers zero_shot.sh and few_shot.sh pin scripts and arguments

##### Elements
- Clear labels: “Dataset Adapter (data/mvtec.py, dataset.py)”, “open_clip (src/open_clip/)”, “WinCLIP head (coca_model.py, loss.py)”, “Trainer (src/training/train.py)”, “Zero-Shot Runner (zero_shot.sh → zero_shot.py)”, “Few-Shot Runner (few_shot.sh → few_shot.py)”

##### Detail
- Major module and its role.
- Connections with the type of data exchanged (e.g., “torch.Tensor batch”, “JSON config”, “model checkpoint”).
- Essential pieces: model configs JSON, vocabulary file, HF push script.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources"
        DS1["Raw Datasets (MVTec, VisA)"]:::data
    end

    %% Data Adapter & Preprocessing
    subgraph "Data Adapter & Preprocessing"
        DA["Dataset Adapter (data/mvtec.py, data/visa.py, dataset.py)"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        OB["open_clip Backbone\n(src/open_clip/)"]:::model
        AH["WinCLIP Anomaly Head\n(coca_model.py, loss.py)"]:::model
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        TR["Trainer\n(src/training/train.py)"]:::train
        UTIL["Utilities\n(distributed.py, scheduler.py, precision.py, logger.py)"]:::train
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        ZS["Zero-Shot Runner\n(zero_shot.sh, src/training/zero_shot.py)"]:::inference
        FS["Few-Shot Runner\n(few_shot.sh, few_shot.py)"]:::inference
    end

    %% External Integration
    subgraph "External Services"
        PS["HF Push Script\n(src/open_clip/push_to_hf_hub.py)"]:::external
        EH["Hugging Face Hub"]:::external
    end

    %% Data Flow
    DS1 --> DA
    DA -->|"torch.Tensor batch"| OB
    OB -->|"feature embeddings"| AH
    AH -->|"anomaly scores/maps"| TR
    TR --> UTIL
    UTIL --> TR
    TR --> OB
    TR --> AH
    DA --> ZS
    DA --> FS
    OB --> ZS
    AH --> ZS
    OB --> FS
    AH --> FS
    TR -->|"push checkpoint"| PS
    PS --> EH

    %% Click Events
    click DA "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/data/mvtec.py"
    click DA "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/data/visa.py"
    click DA "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/dataset.py"
    click OB "https://github.com/zqhang/accurate-winclip-pytorch/tree/main/src/open_clip/"
    click AH "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/open_clip/coca_model.py"
    click AH "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/open_clip/loss.py"
    click TR "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/training/train.py"
    click UTIL "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/training/distributed.py"
    click UTIL "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/training/scheduler.py"
    click UTIL "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/training/precision.py"
    click UTIL "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/training/logger.py"
    click ZS "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/zero_shot.sh"
    click ZS "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/training/zero_shot.py"
    click FS "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/few_shot.sh"
    click FS "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/few_shot.py"
    click PS "https://github.com/zqhang/accurate-winclip-pytorch/blob/main/src/open_clip/push_to_hf_hub.py"

    %% Styles
    classDef data fill:#f9f,stroke:#333,stroke-width:1px
    classDef model fill:#9f9,stroke:#333,stroke-width:1px
    classDef train fill:#f96,stroke:#333,stroke-width:1px
    classDef inference fill:#6cf,stroke:#333,stroke-width:1px
    classDef external fill:#c9c,stroke:#333,stroke-width:1px
```
