How rulixiang/toco works
TOCO is a research implementation of token-contrast weakly supervised semantic segmentation. Its main workflow trains a ViT/DeiT-based model from VOC or COCO images and image-level class labels, then produces and evaluates segmentation predictions; inference tools also support CAM inspection and CRF-refined outputs. Training and inference entry points are documented in the README, while sampled model, dataset, loss, and post-processing code supports the internal layers shown. Some tool implementations were not sampled, so their detailed wiring is left unspecified.
Main entry points: model_seg_neg.py Segmentation network · dist_train_voc_seg_neg.py VOC training · dist_train_coco_seg_neg.py COCO training · infer_seg_coco_ddp.py COCO segmentation inference
Components 22
Data and augmentation
- VOC data
datasets/voc.py - COCO data
datasets/coco.py - Image transforms
datasets/transforms.py
Segmentation model
- Segmentation network
model/model_seg_neg.py - ViT backbone
model/backbone/vit.py - DeiT backbone
model/backbone/deit.py - Segmentation decoder
model/decoder/conv_head.py
Model training
- VOC training
scripts/dist_train_voc_seg_neg.py - COCO training
scripts/dist_train_coco_seg_neg.py - Training losses
model/losses.py - Learning-rate optimizers
utils/optimizer.py
Inference and analysis
- CAM utilities
utils/camutils.py - VOC segmentation inference
tools/infer_seg_voc.py - COCO segmentation inference
tools/infer_seg_coco_ddp.py - CAM inference
tools/infer_cam.py - CAM evaluation
tools/eval_cam.py - Token similarity maps
tools/get_cls_token_cosine_sim_map.py - Segmentation as CAM
tools/infer_seg_as_cam.py - Dense CRF
utils/dcrf.py - Segmentation metrics
utils/evaluate.py
Other components
- Researcher
- Segmentation predictions
Connections 32
- Researcher → VOC training: starts training
- Researcher → COCO training: starts training
- Researcher → VOC segmentation inference: requests inference
- Researcher → COCO segmentation inference: requests inference
- VOC training → VOC data: loads batches
- COCO training → COCO data: loads batches
- VOC data → Image transforms: augments samples
- COCO data → Image transforms: augments samples
- VOC training → Segmentation network: trains model
- COCO training → Segmentation network: trains model
- Segmentation network → ViT backbone: uses backbone
- Segmentation network → DeiT backbone: uses backbone
- Segmentation network → Segmentation decoder: decodes features
- VOC training → Training losses: computes losses
- COCO training → Training losses: computes losses
- VOC training → Learning-rate optimizers: updates parameters
- COCO training → Learning-rate optimizers: updates parameters
- VOC segmentation inference → Segmentation network: runs model
- COCO segmentation inference → Segmentation network: runs model
- VOC segmentation inference → Segmentation predictions: writes predictions
- COCO segmentation inference → Segmentation predictions: writes predictions
- COCO segmentation inference → Dense CRF: refines predictions
- VOC segmentation inference → Segmentation metrics: evaluates predictions
- COCO segmentation inference → Segmentation metrics: evaluates predictions
- CAM inference → Segmentation network: runs model
- CAM inference → CAM utilities: generates CAMs
- CAM evaluation → CAM utilities: evaluates CAMs
- Token similarity maps → Segmentation network: runs model
- Segmentation as CAM → Segmentation network: runs model
- CAM inference → Segmentation predictions: writes maps
- Token similarity maps → Segmentation predictions: writes maps
- Segmentation as CAM → Segmentation predictions: writes maps