How facebookresearch/dinov3 works
DINOv3 is a reference PyTorch implementation for training self-supervised vision foundation models and loading pretrained backbones and task adapters. The main training workflow prepares augmented multi-crop data, trains student/teacher models with distillation losses, and saves checkpoints; pretrained features also support classification, segmentation, depth, detection, and text-related evaluation or inference. The sampled excerpts establish some training, backbone, and distributed relationships; task-specific evaluation wiring is mostly unsampled, so those links are left unverified.
Main entry points: backbones.py Backbone loading · train.py Training runner · ssl_meta_arch.py Student-teacher model · loaders.py Loaders and transforms
Components 22
Data preparation
- Dataset decoding
dinov3/data/datasets/decoders.py - Loaders and transforms
dinov3/data/loaders.py - Multi-crop augmentation
dinov3/data/augmentations.py
Model foundations
- Vision Transformer
dinov3/models/vision_transformer.py - ConvNeXt
dinov3/models/convnext.py - Transformer layers
dinov3/layers/block.py
Training runtime
- Training runner
dinov3/train/train.py - Configuration
dinov3/configs/config.py - Student-teacher model
dinov3/train/ssl_meta_arch.py - Distillation losses
dinov3/loss/dino_clstoken_loss.py - Training schedules
dinov3/train/cosine_lr_scheduler.py - Distributed runtime
dinov3/distributed/torch_distributed_wrapper.py - FSDP and compile
dinov3/fsdp/ac_compile_parallelize.py - Checkpoint storage
dinov3/checkpointer/checkpointer.py
Evaluation and tasks
- Linear evaluation
dinov3/eval/linear.py - Segmentation
dinov3/eval/segmentation - Depth estimation
dinov3/eval/depth - Object detection
dinov3/eval/detection - Vision-language
dinov3/eval/text
Pretrained model access
- Backbone loading
dinov3/hub/backbones.py - Task adapters
dinov3/hub/depthers.py
Other components
- Researcher
Connections 23
- Researcher → Training runner: starts training
README.md - Training runner → Configuration: loads config
dinov3/train/train.py - Training runner → Loaders and transforms: builds loaders
dinov3/train/train.py - Training runner → Student-teacher model: trains model
dinov3/train/train.py - Training runner → Distributed runtime: uses runtime
dinov3/train/train.py - Training runner → Training schedules: uses schedule
dinov3/train/train.py - Training runner → Checkpoint storage: saves state
dinov3/train/train.py - Loaders and transforms → Dataset decoding: loads samples
- Loaders and transforms → Multi-crop augmentation: transforms samples
- Student-teacher model → Vision Transformer: builds backbone
dinov3/train/ssl_meta_arch.py - Student-teacher model → ConvNeXt: builds backbone
- Student-teacher model → Distillation losses: uses objectives
dinov3/train/ssl_meta_arch.py - Student-teacher model → FSDP and compile: configures parallelism
dinov3/train/ssl_meta_arch.py - Vision Transformer → Transformer layers: uses blocks
dinov3/models/vision_transformer.py - Researcher → Backbone loading: loads pretrained
README.md - Backbone loading → Vision Transformer: constructs model
dinov3/hub/backbones.py - Backbone loading → ConvNeXt: constructs model
dinov3/hub/backbones.py - Backbone loading → Task adapters: loads adapters
dinov3/hub/depthers.py - Backbone loading → Linear evaluation: supports probing
README.md - Backbone loading → Segmentation: supports evaluation
README.md - Backbone loading → Depth estimation: supports evaluation
README.md - Backbone loading → Object detection: supports tasks
- Backbone loading → Vision-language: supports tasks