# 7hhhhh/visualad architecture

> Architecture diagram of the GitHub repository 7hhhhh/visualad, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/7hhhhh/visualad
- Repository: https://github.com/7hhhhh/visualad

## Overview

1. **Repository purpose and execution flow**
   - This is a PyTorch repo for zero-shot anomaly detection with a frozen Vision Transformer backbone.
   - The main runtime entry points are `train.py` and `test.py`, with `scripts/CLIP.sh` orchestrating the common cross-dataset pipeline used in the README.
   - Pretrained checkpoints live under `weight/`, so the repo supports both training and direct evaluation.

2. **Data preparation boundary**
   - Raw datasets are not bundled here; the repo expects the same folder/JSON convention as AnomalyCLIP.
   - `generate_dataset_json/` contains one script per benchmark dataset and is responsible for producing dataset metadata JSON files.
   - `dataset.py` is the bridge from those JSON descriptors to PyTorch dataset objects consumed by training and evaluation.

3. **Core model package**
   - `VisualAD_lib/VisualAD.py` appears to define the main VisualAD model.
   - `build_model.py` assembles the backbone, token logic, and anomaly-scoring components.
   - `model_load.py` handles checkpoint loading and model restoration, which matters for cross-dataset evaluation.
   - `constants.py` centralizes model- and dataset-related settings, while `__init__.py` exposes the package API.

4. **Text-free tokenized ViT architecture**
   - The model is built around a frozen ViT backbone, with two learnable tokens: one representing normality and one representing anomaly.
   - The repo is explicitly language-free, so there is no text encoder path; all reasoning happens in visual feature space.
   - `VisualAD_lib/simple_tokenizer.py`, `bpe_simple_vocab_16e6.txt.gz`, and `transform.py` are present because the codebase is CLIP-compatible, even though VisualAD removes the text branch.

5. **Backbone adaptation layer**
   - `utils/backbone_config.py` is the main boundary for supporting multiple backbones.
   - The README indicates two supported backbones: CLIP ViT-L/14@336px and DINOv2 ViT-L/14.
   - This layer is where backbone-specific feature shapes, preprocessing, and loading behavior are normalized so the rest of the model can stay shared.

6. **Feature interaction modules**
   - `utils/spatial_cross_attention.py` corresponds to the Spatial-Aware Cross-Attention idea described in the README.
   - `utils/feature_transform.py`, `utils/normalization.py`, and `utils/transforms.py` likely handle token/patch feature reshaping, recalibration, and preprocessing.
   - `utils/anomaly_detection.py` and `utils/scoring.py` appear to convert refined features into image-level and pixel-level anomaly scores.

7. **Training subsystem**
   - `train.py` likely drives the standard optimization loop using `utils/training_utils.py`.
   - `utils/loss.py` defines the training objectives for separating normal and anomalous representations.
   - `utils/optim/` contains optimizer setup, while `utils/logger.py` handles experiment logging and `utils/metrics.py` tracks performance during training/validation.

8. **Inference and evaluation subsystem**
   - `test.py` is the evaluation entry point, using trained or pretrained weights to produce anomaly predictions.
   - Evaluation output is fed through `utils/metrics.py` for image-level AUROC/F1/AP and pixel-level AUROC/F1/AP/PRO, matching the metrics shown in the README.
   - `utils/visualization.py` supports qualitative inspection of anomaly heatmaps and masks.

9. **Analysis and debugging support**
   - `utils/analysis.py` suggests some offline analysis of scores, features, or failure cases.
   - These utilities are separate from the core training loop, indicating the repo keeps experimentation/diagnostics out of the model path.

10. **Dataset coverage and cross-domain design**
   - The codebase is organized around multiple industrial and medical benchmarks: MVTec, VisA, BTAD, KSDD2, DAGM, DTD-Synthetic, OCT17, BrainMRI, Brain_AD, HIS, CVC-ClinicDB, Endo, and Kvasir.
   - The architecture is designed for cross-dataset zero-shot transfer: train on one source dataset, evaluate on another without task-specific adaptation.
   - The presence of per-dataset JSON generators and shared dataset loading suggests a uniform dataset contract across very different domains.

11. **External dependencies and runtime boundary**
   - The main stack is PyTorch with torchvision and timm, plus scikit-learn, scipy, and tqdm for metrics and utilities.
   - Checkpoints under `weight/train_on_mvtec/CLIP.pth` and `weight/train_on_visa/CLIP.pth` are the repo’s shipped model artifacts.
   - The project depends on external backbone ecosystems, especially CLIP and DINOv2, but wraps them so the anomaly-detection logic remains backbone-agnostic.

12. **How data flows through the system**
   - Dataset JSON and image folders are read by `dataset.py`, transformed, and batched into the training/eval loop.
   - Batches pass through the frozen ViT backbone and the learnable token modules in `VisualAD_lib`.
   - Spatial and feature recalibration modules refine patch-level evidence before scoring.
   - Final outputs are anomaly scores and heatmaps, which are then measured by the metrics and optionally visualized.

Main entry points: [`train.py`](https://github.com/7hhhhh/visualad/blob/main/train.py) (train.py), [`test.py`](https://github.com/7hhhhh/visualad/blob/main/test.py) (test.py), [`VisualAD_lib/VisualAD.py`](https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/VisualAD.py) (VisualAD), [`dataset.py`](https://github.com/7hhhhh/visualad/blob/main/dataset.py) (dataset.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entrypoints["Entry points"]
  node_train_py["train.py<br/>trainer<br/>[train.py]"]
  node_test_py["test.py<br/>evaluator<br/>[test.py]"]
  node_clip_sh["CLIP.sh<br/>pipeline script<br/>[CLIP.sh]"]
end

subgraph group_data["Data pipeline"]
  node_dataset_py["dataset.py<br/>dataset bridge<br/>[dataset.py]"]
  node_gen_json["JSON generators<br/>metadata scripts"]
end

subgraph group_model["VisualAD core"]
  node_visualad_pkg["VisualAD_lib<br/>model package"]
  node_visualad_model["VisualAD<br/>main model<br/>[VisualAD.py]"]
  node_build_model["build_model.py<br/>assembler<br/>[build_model.py]"]
  node_model_load["model_load.py<br/>checkpoint loader<br/>[model_load.py]"]
end

subgraph group_runtime["Runtime utils"]
  node_backbone_config["backbone_config.py<br/>backbone adapter<br/>[backbone_config.py]"]
  node_spatial_ca["spatial attention<br/>feature interaction"]
  node_feature_ops["feature ops<br/>tensor transforms"]
  node_scoring["scoring<br/>[scoring.py]"]
  node_metrics["metrics<br/>evaluation<br/>[metrics.py]"]
  node_training_utils["training utils<br/>optimization loop<br/>[training_utils.py]"]
  node_loss["loss.py<br/>objective<br/>[loss.py]"]
  node_visualization["visualization<br/>inspection<br/>[visualization.py]"]
  node_analysis["analysis<br/>debugging<br/>[analysis.py]"]
end

subgraph group_artifacts["Weights"]
  node_weights_mvtec[("CLIP.pth<br/>checkpoint<br/>[CLIP.pth]")]
  node_weights_visa[("CLIP.pth<br/>checkpoint<br/>[CLIP.pth]")]
end

node_clip_sh -->|"runs"| node_train_py
node_clip_sh -->|"evaluates"| node_test_py
node_train_py -->|"loads"| node_dataset_py
node_test_py -->|"loads"| node_dataset_py
node_dataset_py -.->|"expects"| node_gen_json
node_train_py -->|"trains"| node_visualad_pkg
node_test_py -->|"inferences"| node_visualad_pkg
node_visualad_pkg -->|"contains"| node_visualad_model
node_visualad_model -->|"assembled by"| node_build_model
node_build_model -->|"adapts"| node_backbone_config
node_visualad_model -->|"refines"| node_spatial_ca
node_visualad_model -->|"transforms"| node_feature_ops
node_visualad_model -->|"produces"| node_scoring
node_model_load -->|"restores"| node_visualad_model
node_train_py -->|"uses"| node_training_utils
node_train_py -->|"optimizes"| node_loss
node_test_py -->|"measures"| node_metrics
node_train_py -->|"tracks"| node_metrics
node_test_py -.->|"renders"| node_visualization
node_analysis -.->|"inspects"| node_scoring
node_train_py -.->|"reads"| node_weights_mvtec
node_train_py -.->|"reads"| node_weights_visa
node_test_py -.->|"evaluates"| node_weights_mvtec
node_test_py -.->|"evaluates"| node_weights_visa

click node_train_py "https://github.com/7hhhhh/visualad/blob/main/train.py"
click node_test_py "https://github.com/7hhhhh/visualad/blob/main/test.py"
click node_clip_sh "https://github.com/7hhhhh/visualad/blob/main/scripts/CLIP.sh"
click node_dataset_py "https://github.com/7hhhhh/visualad/blob/main/dataset.py"
click node_gen_json "https://github.com/7hhhhh/visualad/tree/main/generate_dataset_json"
click node_visualad_pkg "https://github.com/7hhhhh/visualad/tree/main/VisualAD_lib"
click node_visualad_model "https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/VisualAD.py"
click node_build_model "https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/build_model.py"
click node_model_load "https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/model_load.py"
click node_backbone_config "https://github.com/7hhhhh/visualad/blob/main/utils/backbone_config.py"
click node_spatial_ca "https://github.com/7hhhhh/visualad/blob/main/utils/spatial_cross_attention.py"
click node_feature_ops "https://github.com/7hhhhh/visualad/blob/main/utils/feature_transform.py"
click node_scoring "https://github.com/7hhhhh/visualad/blob/main/utils/scoring.py"
click node_metrics "https://github.com/7hhhhh/visualad/blob/main/utils/metrics.py"
click node_training_utils "https://github.com/7hhhhh/visualad/blob/main/utils/training_utils.py"
click node_loss "https://github.com/7hhhhh/visualad/blob/main/utils/loss.py"
click node_visualization "https://github.com/7hhhhh/visualad/blob/main/utils/visualization.py"
click node_analysis "https://github.com/7hhhhh/visualad/blob/main/utils/analysis.py"
click node_weights_mvtec "https://github.com/7hhhhh/visualad/blob/main/weight/train_on_mvtec/CLIP.pth"
click node_weights_visa "https://github.com/7hhhhh/visualad/blob/main/weight/train_on_visa/CLIP.pth"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_py,node_test_py,node_clip_sh toneBlue
class node_dataset_py,node_gen_json toneAmber
class node_visualad_pkg,node_visualad_model,node_build_model,node_model_load toneMint
class node_backbone_config,node_spatial_ca,node_feature_ops,node_scoring,node_metrics,node_training_utils,node_loss,node_visualization,node_analysis toneRose
class node_weights_mvtec,node_weights_visa toneIndigo
```

## Components

### Entry points

Training, evaluation, and orchestration scripts.

- [train.py](https://github.com/7hhhhh/visualad/blob/main/train.py): trainer · Main training entry point. · `train.py`
- [test.py](https://github.com/7hhhhh/visualad/blob/main/test.py): evaluator · Main evaluation entry point. · `test.py`
- [CLIP.sh](https://github.com/7hhhhh/visualad/blob/main/scripts/CLIP.sh): pipeline script · Orchestrates the common cross-dataset run. · `scripts/CLIP.sh`

### Data pipeline

Dataset contract and benchmark JSON generators.

- [dataset.py](https://github.com/7hhhhh/visualad/blob/main/dataset.py): dataset bridge · Converts dataset JSON into PyTorch datasets. · `dataset.py`
- [JSON generators](https://github.com/7hhhhh/visualad/tree/main/generate_dataset_json): metadata scripts · Per-benchmark scripts that build dataset descriptors. · `generate_dataset_json`

### VisualAD core

Model package, backbone adaptation, and checkpoint loading.

- [VisualAD_lib](https://github.com/7hhhhh/visualad/tree/main/VisualAD_lib): model package · Core VisualAD implementation and package API. · `VisualAD_lib`
- [VisualAD](https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/VisualAD.py): main model · Defines the tokenized zero-shot anomaly detector. · `VisualAD_lib/VisualAD.py`
- [build_model.py](https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/build_model.py): assembler · Constructs backbone, tokens, and scoring path. · `VisualAD_lib/build_model.py`
- [model_load.py](https://github.com/7hhhhh/visualad/blob/main/VisualAD_lib/model_load.py): checkpoint loader · Restores pretrained or trained weights. · `VisualAD_lib/model_load.py`

### Runtime utils

Training, scoring, metrics, visualization, and analysis support.

- [backbone_config.py](https://github.com/7hhhhh/visualad/blob/main/utils/backbone_config.py): backbone adapter · Normalizes CLIP and DINOv2 backbone behavior. · `utils/backbone_config.py`
- [spatial attention](https://github.com/7hhhhh/visualad/blob/main/utils/spatial_cross_attention.py): feature interaction · Spatial-aware cross-attention refinement. · `utils/spatial_cross_attention.py`
- [feature ops](https://github.com/7hhhhh/visualad/blob/main/utils/feature_transform.py): tensor transforms · Feature reshaping and recalibration helpers. · `utils/feature_transform.py`
- [scoring](https://github.com/7hhhhh/visualad/blob/main/utils/scoring.py): anomaly scoring · Converts refined features into anomaly scores. · `utils/scoring.py`
- [metrics](https://github.com/7hhhhh/visualad/blob/main/utils/metrics.py): evaluation · Computes AUROC, AP, F1, and PRO-style scores. · `utils/metrics.py`
- [training utils](https://github.com/7hhhhh/visualad/blob/main/utils/training_utils.py): optimization loop · Shared training loop helpers and batch handling. · `utils/training_utils.py`
- [loss.py](https://github.com/7hhhhh/visualad/blob/main/utils/loss.py): objective · Training objectives for normal/anomalous separation. · `utils/loss.py`
- [visualization](https://github.com/7hhhhh/visualad/blob/main/utils/visualization.py): inspection · Heatmap and mask visualization utilities. · `utils/visualization.py`
- [analysis](https://github.com/7hhhhh/visualad/blob/main/utils/analysis.py): debugging · Offline score and failure-case analysis. · `utils/analysis.py`

### Weights

Shipped pretrained checkpoints.

- [CLIP.pth](https://github.com/7hhhhh/visualad/blob/main/weight/train_on_mvtec/CLIP.pth): checkpoint · Shipped checkpoint for MVTec training. · `weight/train_on_mvtec/CLIP.pth`
- [CLIP.pth](https://github.com/7hhhhh/visualad/blob/main/weight/train_on_visa/CLIP.pth): checkpoint · Shipped checkpoint for VisA training. · `weight/train_on_visa/CLIP.pth`

## Connections

- CLIP.sh → train.py: runs · Common training pipeline entry.
- CLIP.sh → test.py: evaluates · Common evaluation pipeline entry.
- train.py → dataset.py: loads · Reads dataset JSON and image folders.
- test.py → dataset.py: loads · Builds evaluation datasets from the same contract.
- dataset.py → JSON generators: expects · Consumes benchmark-specific JSON metadata.
- train.py → VisualAD_lib: trains · Optimizes the core VisualAD model.
- test.py → VisualAD_lib: inferences · Runs the model for anomaly prediction.
- VisualAD_lib → VisualAD: contains · Package exposes the main model implementation.
- VisualAD → build_model.py: assembled by · Model construction is centralized here.
- build_model.py → backbone_config.py: adapts · Selects and normalizes supported backbones.
- VisualAD → spatial attention: refines · Applies spatial-aware feature interaction.
- VisualAD → feature ops: transforms · Shapes token and patch features for scoring.
- VisualAD → scoring: produces · Outputs anomaly scores and maps.
- model_load.py → VisualAD: restores · Loads checkpoints into the model.
- train.py → training utils: uses · Shared optimization and batching helpers.
- train.py → loss.py: optimizes · Training objective for separation.
- test.py → metrics: measures · Computes evaluation metrics.
- train.py → metrics: tracks · Monitors validation performance.
- test.py → visualization: renders · Supports qualitative inspection.
- analysis → scoring: inspects · Offline analysis of scores and failures.
- train.py → CLIP.pth: reads · Can initialize from shipped checkpoint.
- train.py → CLIP.pth: reads · Can initialize from shipped checkpoint.
- test.py → CLIP.pth: evaluates · Direct evaluation from pretrained weights.
- test.py → CLIP.pth: evaluates · Direct evaluation from pretrained weights.
