How gzaraunitn/autolabel works
Project Type and Purpose
• This is an academic / research-oriented machine-learning framework for open-set video domain adaptation.
• It uses a CLIP backbone to extract visual–text features and adapts across video datasets.
File Structure
Top-level directories:
• clip/ ← CLIP backbone, tokenizer, model zoo
• configs/ ← global + per-setting YAMLs
• data/ ← CSVs, transforms, dataset & dataloader code
• txt/ ← dataset splits (source/target video lists)
• modules/ ← custom CLIP modules, loss functions, prompts
• loops/ ← training_step.py & test_step.py (the engine)
• utils/ ← schedulers, solvers, attribute JSONs, misc utils
• scripts/ & train.py ← experiment entry points
Key patterns:
– Modular separation: data ↔ model ↔ training loops ↔ utils
– Config-driven experiment runner (train.py + autolabel.yaml + configs/)
– Core ML flow in loops/ + modules/
Readme Insights
• Single-script experiments: a user picks a “setting” YAML and calls train.py or scripts/run.sh
• Data paths must be configured inside data/dataloader.py
• Dependencies (PyTorch, CLIP, etc.) defined in autolabel.yaml
Main Components & Interactions
a. Experiment Runner
– Entry point: scripts/run.sh or train.py
– Parses CLI flags, loads a config YAML
b. Configuration
– configs/config.yaml + configs/setting/*.yaml drive dataset paths, hyperparams
c. Data Ingestion
– txt/ holds file lists per domain
– data/data_utils.py & data/custom_transforms.py define preprocessing
– data/dataset.py builds a torch.utils.data.Dataset
– data/dataloader.py wraps with DataLoader
d. Model
– clip/ module handles loading CLIP backbone, tokenizer, pre‐trained weights
– modules/clip_modules.py augments CLIP for video frames or features
– modules/prompts.py builds text prompts for open-set classes
– modules/kll_loss.py computes the open-set adaptation loss
e. Training Engine
– loops/training_step.py encapsulates one train iteration: forward, loss, backward
– loops/test_step.py runs validation or domain adaptation evaluation
f. Optimization & Scheduling
– utils/solver.py configures optimizers (SGD/Adam)
– utils/lr_scheduler.py sets learning-rate schedules
g. Outputs
– Checkpoints, logs, validation metrics (written to disk / console)
Data Flow (simplified):
[Runner] → load [Config] → init [Dataset/Dataloader] → init [Model + Prompts]
→ init [Optimizer + Scheduler]
→ loop: (Dataloader → Model → Loss → Optimizer.step → Scheduler.step)
→ periodically call TestStep → write metrics & checkpoints
Architectural Patterns & Principles
• Separation of concerns: keep data, model, training, utils in distinct packages
• Configuration-first design: no hard-coded paths or hyperparams
• Single responsibility: each file/class does one job (e.g., kll_loss only computes its loss)
Detailed Elements
• Config loader → YAML files
• Data pipeline: txt files → custom_transforms → Dataset → DataLoader
• Model stack: CLIP backbone → prompt module → projection heads
• Loss computation: kll_loss
• Optimizer & LR scheduler
• Training loop vs. test loop
• External resources: disk paths, pre-trained weights
• Output artifacts: model checkpoints, metrics