How zqhang/anomalyclip works
Project type and purpose
• This is an open-source research codebase for zero-shot anomaly detection, named AnomalyCLIP.
• Purpose: Adapt a pre-trained vision-language model (CLIP) with object-agnostic text prompts to detect and segment anomalies across diverse industrial and medical imaging datasets without requiring target-domain training samples.
File structure
Top‐level layout:
• AnomalyCLIP_lib/ ← Core library implementing model building, CLIP wrapper, tokenizers, transforms, prompt learning
• generate_dataset_json/ ← Dataset-specific JSON generators for all supported datasets
• train.py, test.py, test_one_example.py ← High-level scripts orchestrating training and inference
• train.sh, test.sh, test_one_example.sh ← Shell wrappers for reproducibility
• dataset.py ← Registry of dataset metadata
• loss.py, metrics.py ← Custom loss and evaluation metrics
• prompt_ensemble.py ← Ensemble logic for multiple prompt templates
• logger.py ← Logging utility
• visualization.py ← Utility to render anomaly maps and plots
• utils.py ← Miscellaneous helper functions
• checkpoints/, results/ ← Model checkpoints and experiment outputs
• assets/overview.pdf ← High-level architecture overview
Architectural patterns:
• Modular separation of data preparation, core model library, training pipeline, evaluation & visualization.
• Script-driven orchestration (train/test), library functions for model internals.
Key insights from README
• Framework: PyTorch 2.0.0
• Relies on an external open_clip or built-in CLIP implementation (AnomalyCLIP_lib/CLIP.py).
• Workflow:
a. Download raw images →
b. Run generate_dataset_json to produce a unified JSON manifest →
c. train.py uses manifest + AnomalyCLIP_lib to build and train model once on MVTec AD →
d. test.py zero-shot tests on all other datasets →
e. visualization and logging produce output artifact.
Final checklist
• Explicit callouts for key files/scripts (train.py, test.sh, generate_dataset_json/*)
• Notes on external dependencies (PyTorch, CLIP, tokenizers)