How zqhang/accurate-winclip-pytorch works
Project type and purpose
• This is an open-source machine-learning toolkit (“WinCLIP” / “AnomalyCLIP”) for zero-shot and few-shot anomaly detection/segmentation built on top of the OpenAI-inspired CLIP model (via the open_clip codebase).
• It provides data ingestion from standard anomaly datasets (MVTec, VisA), wraps an open_clip CLIP backbone, implements WinCLIP anomaly scoring, and delivers evaluation scripts.
File structure
• Top-level scripts:
• zero_shot.sh, few_shot.sh, reproduce_WinCLIP.py
• test.py (smoke tests)
• Data layer:
• data/ (dataset adapters: mvtec.py, visa.py)
• dataset.py (unified dataset interface)
• Model layer:
• src/open_clip/ (CLIP implementation and wrappers, model configs, tokenizers, generation utilities)
• Training & inference:
• src/training/ – data loaders, distributed training, scheduler, precision, logger, main entrypoints (train.py, zero_shot.py, few_shot.py)
• Configuration & dependencies:
• requirements.txt
• bpe-vocabulary, JSON model configs
• External integration:
• push_to_hf_hub.py (Hugging Face Hub)
Core components
a. Data ingestion & preprocessing
b. Model backbone
• “open_clip backbone” (ViT/ResNet models, tokenizer, transform)
• “WinCLIP anomaly head” (coca_model.py, generation_utils.py, loss.py)
c. Training pipeline
• “Training orchestrator” (train.py) → modules: distributed.py, scheduler.py, precision.py, logger.py
d. Zero-Shot & Few-Shot inference pipelines
• “Zero-shot runner” (zero_shot.py + zero_shot.sh)
• “Few-shot runner” (few_shot.py + few_shot.sh)
e. External services
• “Hugging Face Hub” (push_to_hf_hub.py)
Relationships & data flows
• Data → Dataset adapter → Preprocessed tensor batches
• Batches → Model backbone → Feature embeddings
• Embeddings → Anomaly scoring head → Anomaly maps / labels
• Training orchestrator ↔ scheduler/precision/logger → updates Model backbone weights
• Inference runners call model in eval mode, write metrics to disk
• Optionally push trained weights to HF Hub
Architectural patterns & principles
• Modular separation: Data layer, Model layer, Training layer, Inference layer
• Config-driven: All backbones and hyperparameters are in JSON and params.py
• Extensibility: Add new datasets by dropping adapters in data/, add new backbones by supplying JSON in model_configs/
• Reproducibility: Shell wrappers zero_shot.sh and few_shot.sh pin scripts and arguments
Elements
• Clear labels: “Dataset Adapter (data/mvtec.py, dataset.py)”, “open_clip (src/open_clip/)”, “WinCLIP head (coca_model.py, loss.py)”, “Trainer (src/training/train.py)”, “Zero-Shot Runner (zero_shot.sh → zero_shot.py)”, “Few-Shot Runner (few_shot.sh → few_shot.py)”
Detail
• Major module and its role.
• Connections with the type of data exchanged (e.g., “torch.Tensor batch”, “JSON config”, “model checkpoint”).
• Essential pieces: model configs JSON, vocabulary file, HF push script.