# rulixiang/toco architecture

> Architecture diagram of the GitHub repository rulixiang/toco, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rulixiang/toco
- Repository: https://github.com/rulixiang/toco

## Overview

TOCO is a research implementation of token-contrast weakly supervised semantic segmentation. Its main workflow trains a ViT/DeiT-based model from VOC or COCO images and image-level class labels, then produces and evaluates segmentation predictions; inference tools also support CAM inspection and CRF-refined outputs. Training and inference entry points are documented in the README, while sampled model, dataset, loss, and post-processing code supports the internal layers shown. Some tool implementations were not sampled, so their detailed wiring is left unspecified.

Main entry points: [`model/model_seg_neg.py`](https://github.com/rulixiang/toco/blob/main/model/model_seg_neg.py) (Segmentation network), [`scripts/dist_train_voc_seg_neg.py`](https://github.com/rulixiang/toco/blob/main/scripts/dist_train_voc_seg_neg.py) (VOC training), [`scripts/dist_train_coco_seg_neg.py`](https://github.com/rulixiang/toco/blob/main/scripts/dist_train_coco_seg_neg.py) (COCO training), [`tools/infer_seg_coco_ddp.py`](https://github.com/rulixiang/toco/blob/main/tools/infer_seg_coco_ddp.py) (COCO segmentation inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data and augmentation"]
  node_voc["VOC data<br/>[voc.py]"]
  node_coco["COCO data<br/>[coco.py]"]
  node_transforms["Image transforms<br/>[transforms.py]"]
end

subgraph group_model["Segmentation model"]
  node_network["Segmentation network<br/>[model_seg_neg.py]"]
  node_vit["ViT backbone<br/>[vit.py]"]
  node_deit["DeiT backbone<br/>[deit.py]"]
  node_decoder["Segmentation decoder<br/>[conv_head.py]"]
end

subgraph group_training["Model training"]
  node_trainvoc["VOC training"]
  node_traincoco["COCO training"]
  node_losses["Training losses<br/>[losses.py]"]
  node_optimizer["Learning-rate optimizers<br/>[optimizer.py]"]
end

subgraph group_inference["Inference and analysis"]
  node_camutils["CAM utilities<br/>[camutils.py]"]
  node_infer_voc["VOC segmentation inference<br/>[infer_seg_voc.py]"]
  node_infer_coco["COCO segmentation inference"]
  node_infer_cam["CAM inference<br/>[infer_cam.py]"]
  node_eval_cam["CAM evaluation<br/>[eval_cam.py]"]
  node_cosine_map["Token similarity maps"]
  node_seg_as_cam["Segmentation as CAM"]
  node_crf["Dense CRF<br/>[dcrf.py]"]
  node_evaluation["Segmentation metrics<br/>[evaluate.py]"]
end

node_engineer(("Researcher"))
node_predictions["Segmentation predictions"]

node_engineer -->|"starts training"| node_trainvoc
node_engineer -->|"starts training"| node_traincoco
node_engineer -->|"requests inference"| node_infer_voc
node_engineer -->|"requests inference"| node_infer_coco
node_trainvoc -->|"loads batches"| node_voc
node_traincoco -->|"loads batches"| node_coco
node_voc -->|"augments samples"| node_transforms
node_coco -->|"augments samples"| node_transforms
node_trainvoc -->|"trains model"| node_network
node_traincoco -->|"trains model"| node_network
node_network -->|"uses backbone"| node_vit
node_network -->|"uses backbone"| node_deit
node_network -->|"decodes features"| node_decoder
node_trainvoc -->|"computes losses"| node_losses
node_traincoco -->|"computes losses"| node_losses
node_trainvoc -->|"updates parameters"| node_optimizer
node_traincoco -->|"updates parameters"| node_optimizer
node_infer_voc -->|"runs model"| node_network
node_infer_coco -->|"runs model"| node_network
node_infer_voc -->|"writes predictions"| node_predictions
node_infer_coco -->|"writes predictions"| node_predictions
node_infer_coco -->|"refines predictions"| node_crf
node_infer_voc -->|"evaluates predictions"| node_evaluation
node_infer_coco -->|"evaluates predictions"| node_evaluation
node_infer_cam -->|"runs model"| node_network
node_infer_cam -->|"generates CAMs"| node_camutils
node_eval_cam -->|"evaluates CAMs"| node_camutils
node_cosine_map -->|"runs model"| node_network
node_seg_as_cam -->|"runs model"| node_network
node_infer_cam -->|"writes maps"| node_predictions
node_cosine_map -->|"writes maps"| node_predictions
node_seg_as_cam -->|"writes maps"| node_predictions

click node_voc "https://github.com/rulixiang/toco/blob/main/datasets/voc.py"
click node_coco "https://github.com/rulixiang/toco/blob/main/datasets/coco.py"
click node_transforms "https://github.com/rulixiang/toco/blob/main/datasets/transforms.py"
click node_trainvoc "https://github.com/rulixiang/toco/blob/main/scripts/dist_train_voc_seg_neg.py"
click node_traincoco "https://github.com/rulixiang/toco/blob/main/scripts/dist_train_coco_seg_neg.py"
click node_network "https://github.com/rulixiang/toco/blob/main/model/model_seg_neg.py"
click node_vit "https://github.com/rulixiang/toco/blob/main/model/backbone/vit.py"
click node_deit "https://github.com/rulixiang/toco/blob/main/model/backbone/deit.py"
click node_decoder "https://github.com/rulixiang/toco/blob/main/model/decoder/conv_head.py"
click node_losses "https://github.com/rulixiang/toco/blob/main/model/losses.py"
click node_optimizer "https://github.com/rulixiang/toco/blob/main/utils/optimizer.py"
click node_camutils "https://github.com/rulixiang/toco/blob/main/utils/camutils.py"
click node_infer_voc "https://github.com/rulixiang/toco/blob/main/tools/infer_seg_voc.py"
click node_infer_coco "https://github.com/rulixiang/toco/blob/main/tools/infer_seg_coco_ddp.py"
click node_infer_cam "https://github.com/rulixiang/toco/blob/main/tools/infer_cam.py"
click node_eval_cam "https://github.com/rulixiang/toco/blob/main/tools/eval_cam.py"
click node_cosine_map "https://github.com/rulixiang/toco/blob/main/tools/get_cls_token_cosine_sim_map.py"
click node_seg_as_cam "https://github.com/rulixiang/toco/blob/main/tools/infer_seg_as_cam.py"
click node_crf "https://github.com/rulixiang/toco/blob/main/utils/dcrf.py"
click node_evaluation "https://github.com/rulixiang/toco/blob/main/utils/evaluate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_voc,node_coco,node_transforms toneBlue
class node_network,node_vit,node_deit,node_decoder toneAmber
class node_trainvoc,node_traincoco,node_losses,node_optimizer toneMint
class node_camutils,node_infer_voc,node_infer_coco,node_infer_cam,node_eval_cam,node_cosine_map,node_seg_as_cam,node_crf,node_evaluation toneRose
class node_engineer,node_predictions toneIndigo
```

## Components

### Data and augmentation

- [VOC data](https://github.com/rulixiang/toco/blob/main/datasets/voc.py): `datasets/voc.py`
- [COCO data](https://github.com/rulixiang/toco/blob/main/datasets/coco.py): `datasets/coco.py`
- [Image transforms](https://github.com/rulixiang/toco/blob/main/datasets/transforms.py): `datasets/transforms.py`

### Segmentation model

- [Segmentation network](https://github.com/rulixiang/toco/blob/main/model/model_seg_neg.py): `model/model_seg_neg.py`
- [ViT backbone](https://github.com/rulixiang/toco/blob/main/model/backbone/vit.py): `model/backbone/vit.py`
- [DeiT backbone](https://github.com/rulixiang/toco/blob/main/model/backbone/deit.py): `model/backbone/deit.py`
- [Segmentation decoder](https://github.com/rulixiang/toco/blob/main/model/decoder/conv_head.py): `model/decoder/conv_head.py`

### Model training

- [VOC training](https://github.com/rulixiang/toco/blob/main/scripts/dist_train_voc_seg_neg.py): `scripts/dist_train_voc_seg_neg.py`
- [COCO training](https://github.com/rulixiang/toco/blob/main/scripts/dist_train_coco_seg_neg.py): `scripts/dist_train_coco_seg_neg.py`
- [Training losses](https://github.com/rulixiang/toco/blob/main/model/losses.py): `model/losses.py`
- [Learning-rate optimizers](https://github.com/rulixiang/toco/blob/main/utils/optimizer.py): `utils/optimizer.py`

### Inference and analysis

- [CAM utilities](https://github.com/rulixiang/toco/blob/main/utils/camutils.py): `utils/camutils.py`
- [VOC segmentation inference](https://github.com/rulixiang/toco/blob/main/tools/infer_seg_voc.py): `tools/infer_seg_voc.py`
- [COCO segmentation inference](https://github.com/rulixiang/toco/blob/main/tools/infer_seg_coco_ddp.py): `tools/infer_seg_coco_ddp.py`
- [CAM inference](https://github.com/rulixiang/toco/blob/main/tools/infer_cam.py): `tools/infer_cam.py`
- [CAM evaluation](https://github.com/rulixiang/toco/blob/main/tools/eval_cam.py): `tools/eval_cam.py`
- [Token similarity maps](https://github.com/rulixiang/toco/blob/main/tools/get_cls_token_cosine_sim_map.py): `tools/get_cls_token_cosine_sim_map.py`
- [Segmentation as CAM](https://github.com/rulixiang/toco/blob/main/tools/infer_seg_as_cam.py): `tools/infer_seg_as_cam.py`
- [Dense CRF](https://github.com/rulixiang/toco/blob/main/utils/dcrf.py): `utils/dcrf.py`
- [Segmentation metrics](https://github.com/rulixiang/toco/blob/main/utils/evaluate.py): `utils/evaluate.py`

### Other components

- **Researcher**
- **Segmentation predictions**

## Connections

- Researcher → VOC training: starts training
- Researcher → COCO training: starts training
- Researcher → VOC segmentation inference: requests inference
- Researcher → COCO segmentation inference: requests inference
- VOC training → VOC data: loads batches
- COCO training → COCO data: loads batches
- VOC data → Image transforms: augments samples
- COCO data → Image transforms: augments samples
- VOC training → Segmentation network: trains model
- COCO training → Segmentation network: trains model
- Segmentation network → ViT backbone: uses backbone
- Segmentation network → DeiT backbone: uses backbone
- Segmentation network → Segmentation decoder: decodes features
- VOC training → Training losses: computes losses
- COCO training → Training losses: computes losses
- VOC training → Learning-rate optimizers: updates parameters
- COCO training → Learning-rate optimizers: updates parameters
- VOC segmentation inference → Segmentation network: runs model
- COCO segmentation inference → Segmentation network: runs model
- VOC segmentation inference → Segmentation predictions: writes predictions
- COCO segmentation inference → Segmentation predictions: writes predictions
- COCO segmentation inference → Dense CRF: refines predictions
- VOC segmentation inference → Segmentation metrics: evaluates predictions
- COCO segmentation inference → Segmentation metrics: evaluates predictions
- CAM inference → Segmentation network: runs model
- CAM inference → CAM utilities: generates CAMs
- CAM evaluation → CAM utilities: evaluates CAMs
- Token similarity maps → Segmentation network: runs model
- Segmentation as CAM → Segmentation network: runs model
- CAM inference → Segmentation predictions: writes maps
- Token similarity maps → Segmentation predictions: writes maps
- Segmentation as CAM → Segmentation predictions: writes maps