How atsumiyai/locoop works
Project type and purpose
• This is a research‐oriented PyTorch codebase implementing “LoCoOp: Few‐Shot Out‐of‐Distribution Detection via Prompt Learning.”
File structure
• Top‐level modules:
• clip_w_local/ – CLIP wrapper, tokenizer, vocabulary, model definitions.
• configs/ – YAML files defining datasets and trainer (LoCoOp) hyperparameters.
• datasets/ – custom DataLoader wrappers for in‐distribution datasets (ImageNet, Oxford Pets).
• trainers/ – the LoCoOp trainer logic (train loop, OOD regularization).
• utils/ – helper functions for detection, training/evaluation, plotting.
• scripts/locoop/ – shell scripts to launch training, evaluation, and visualization with specific arguments.
• train.py, eval_ood_detection.py, demo_visualization.py – entry‐point Python scripts.
• figure/ – stored architecture and visualization PNGs.
• External dependencies and build artifacts: requirements.txt, .gitignore, LICENSE.
README for additional insights
• The paper’s high‐level architecture (figure/framework.png) shows how CLIP local features are used for OOD regularization via prompt learning.
• Training, evaluation, and visualization commands are driven by shell scripts reading from configs.
a. Main components
1. Data Storage
• Raw data folder (data/imagenet, data/iNaturalist, etc.)
• YAML dataset configs under configs/datasets
2. Configuration Manager
• YAML files for data and trainer under configs/
• A loader in train.py and eval scripts that parses these configs
3. Preprocessing & Tokenization
• simple_tokenizer.py & BPE vocab
• Data augmentation / transforms (in Dassl.pytorch)
4. CLIP Wrapper
• clip.py and model.py in clip_w_local/ – load pre‐trained CLIP, expose local features
5. Prompt Learning Module
• trainers/locoop.py – implements LoCoOp prompt parameterization and OOD regularization
6. Trainer
• train.py – orchestrates dataset loader, CLIP wrapper, LoCoOp trainer, optimizer, logging
7. Evaluation
• eval_ood_detection.py – loads checkpoint, runs MCM or GL‑MCM inference on OOD datasets
8. Visualization
• demo_visualization.py – extracts and plots OOD regions using plot_util.py
9. External Services & Dependencies
• Dassl.pytorch (for backbone data transforms and base Trainer classes)
• OpenAI CLIP library
• MCM codebase for OOD score computation
b. Relationships and interactions
• Trainer ↔ Configuration Manager (reads hyperparams)
• Trainer → Checkpoint Writer → Stored Models
• Evaluation script reads Stored Models + Data Storage → computes OOD metrics via MCM
• Visualization script reads Stored Models + an image path → outputs attention/heatmaps
• All Python entry‐points invoked via shell scripts in scripts/locoop
c. Architectural patterns & design principles
• Config‐driven pipeline (YAML configs decouple hyperparameters from code)
• Modular separation: data, model definition, training logic, evaluation, visualization
• Wrapper pattern: clip_w_local module abstracts CLIP internals and local feature extraction
• Reuse of external toolbox (Dassl) for common functionality, promoting DRY
d. Key technologies, frameworks, libraries
• PyTorch 1.8.1
• Dassl.pytorch for dataset handling and base training loops
• OpenAI CLIP for vision‐language backbone
• MCM (and optional GL‐MCM) for OOD score computation
• YAML for configuration management
• Shell scripts for reproducible experiment commands
Detail without over‐fragmentation
• Logical component above.
• Avoid splitting too finely (e.g., every single Python file), but ensure each major responsibility is shown—data ingestion, model definition, training loop, evaluation, visualization, and configuration.
• Both static structure (modules and directories) and dynamic flow (training and inference pipelines).