# ermitaju1/starc architecture

> Architecture diagram of the GitHub repository ermitaju1/starc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ermitaju1/starc
- Repository: https://github.com/ermitaju1/starc

## Overview

STaRC is a research system for retrieval-augmented dense video captioning, emphasizing supervised saliency. The principal evidenced workflow is a user launching `dvc_ret.py`, which configures and runs the STaRC model, then invokes captioning and SODA evaluation. Dataset loaders, caption chapter utilities, and evaluation metrics are distinct supporting branches. The index and excerpts establish model internals and evaluator dependencies, but do not show the runner calling dataset loaders; that connection is therefore omitted.

Main entry points: [`dvc_ret.py`](https://github.com/ermitaju1/starc/blob/main/dvc_ret.py) (Training runner), [`model/STaRC.py`](https://github.com/ermitaju1/starc/blob/main/model/STaRC.py) (STaRC model), [`dvc_eval/SODA/soda.py`](https://github.com/ermitaju1/starc/blob/main/dvc_eval/SODA/soda.py) (SODA scoring), [`dvc_eval/pycocoevalcap/cider/cider.py`](https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/cider/cider.py) (CIDEr metric)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Training and inference"]
  node_runner["Training runner<br/>[dvc_ret.py]"]
  node_arguments["Run configuration<br/>[args.py]"]
end

subgraph group_data["Video data"]
  node_chapter_convert["Chapter conversion<br/>[desc2chapters.py]"]
  node_chapter_utils["Chapter utilities<br/>[chapter_utils.py]"]
  node_dvc_data["Dense-caption data<br/>[dvc_dataset.py]"]
  node_dvc_data_alt["Alternate DVC data<br/>[dvc_dataset_2.py]"]
  node_vc_data["Video-caption data<br/>[vc_dataset.py]"]
  node_video_data["Video features<br/>[video_dataset.py]"]
  node_youtube_data["YouTube data<br/>[yt_dataset.py]"]
  node_span_masking["T5 span masking<br/>[t5.py]"]
end

subgraph group_model["Captioning model"]
  node_starc["STaRC model<br/>[STaRC.py]"]
  node_t5_model["T5 generation<br/>[modeling_t5.py]"]
  node_vision["Vision encoder<br/>[vit.py]"]
  node_asot["ASOT segmentation<br/>[asot.py]"]
  node_texttitling["Text titling<br/>[texttitling.py]"]
  node_saliency_loss["Saliency loss<br/>[HD_loss.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_dvc_eval["DVC evaluator<br/>[eval_dvc.py]"]
  node_vc_eval["VC evaluator<br/>[eval_vc.py]"]
  node_soda_entry["SODA evaluation entry<br/>[eval_soda.py]"]
  node_soda["SODA scoring<br/>[soda.py]"]
  node_anet_data["Caption evaluation data<br/>[dataset.py]"]
  node_cider["CIDEr metric<br/>[cider.py]"]
  node_cider_scorer["CIDEr scorer<br/>[cider_scorer.py]"]
  node_meteor["METEOR metric<br/>[meteor.py]"]
end

node_user(("Research user"))

node_user -->|"launches"| node_runner
node_runner -->|"reads options"| node_arguments
node_runner -->|"runs model"| node_starc
node_runner -->|"evaluates captions"| node_dvc_eval
node_runner -->|"evaluates SODA"| node_soda_entry
node_chapter_convert -->|"uses"| node_chapter_utils
node_dvc_data -->|"uses"| node_span_masking
node_dvc_data_alt -->|"uses"| node_span_masking
node_youtube_data -->|"uses"| node_span_masking
node_starc -->|"generates with"| node_t5_model
node_starc -->|"encodes video"| node_vision
node_starc -->|"segments video"| node_asot
node_starc -->|"uses loss"| node_saliency_loss
node_soda_entry -->|"loads data"| node_anet_data
node_soda_entry -->|"runs scorer"| node_soda
node_soda -->|"scores data"| node_anet_data
node_soda -->|"uses metric"| node_cider
node_soda -->|"uses metric"| node_meteor
node_dvc_eval -->|"uses metric"| node_cider
node_dvc_eval -->|"uses metric"| node_meteor
node_vc_eval -->|"uses metric"| node_cider
node_vc_eval -->|"uses metric"| node_meteor
node_cider -->|"delegates scoring"| node_cider_scorer

click node_runner "https://github.com/ermitaju1/starc/blob/main/dvc_ret.py"
click node_arguments "https://github.com/ermitaju1/starc/blob/main/args.py"
click node_chapter_convert "https://github.com/ermitaju1/starc/blob/main/collection/desc2chapters.py"
click node_chapter_utils "https://github.com/ermitaju1/starc/blob/main/collection/chapter_utils.py"
click node_dvc_data "https://github.com/ermitaju1/starc/blob/main/dataset/dvc_dataset.py"
click node_dvc_data_alt "https://github.com/ermitaju1/starc/blob/main/dataset/dvc_dataset_2.py"
click node_vc_data "https://github.com/ermitaju1/starc/blob/main/dataset/vc_dataset.py"
click node_video_data "https://github.com/ermitaju1/starc/blob/main/dataset/video_dataset.py"
click node_youtube_data "https://github.com/ermitaju1/starc/blob/main/dataset/yt_dataset.py"
click node_span_masking "https://github.com/ermitaju1/starc/blob/main/util/t5.py"
click node_starc "https://github.com/ermitaju1/starc/blob/main/model/STaRC.py"
click node_t5_model "https://github.com/ermitaju1/starc/blob/main/model/modeling_t5.py"
click node_vision "https://github.com/ermitaju1/starc/blob/main/model/vit.py"
click node_asot "https://github.com/ermitaju1/starc/blob/main/model/asot.py"
click node_texttitling "https://github.com/ermitaju1/starc/blob/main/model/texttitling.py"
click node_saliency_loss "https://github.com/ermitaju1/starc/blob/main/util/HD_loss.py"
click node_dvc_eval "https://github.com/ermitaju1/starc/blob/main/dvc_eval/eval_dvc.py"
click node_vc_eval "https://github.com/ermitaju1/starc/blob/main/dvc_eval/eval_vc.py"
click node_soda_entry "https://github.com/ermitaju1/starc/blob/main/dvc_eval/eval_soda.py"
click node_soda "https://github.com/ermitaju1/starc/blob/main/dvc_eval/SODA/soda.py"
click node_anet_data "https://github.com/ermitaju1/starc/blob/main/dvc_eval/SODA/dataset.py"
click node_cider "https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/cider/cider.py"
click node_cider_scorer "https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/cider/cider_scorer.py"
click node_meteor "https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/meteor/meteor.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_arguments,node_user toneBlue
class node_chapter_convert,node_chapter_utils,node_dvc_data,node_dvc_data_alt,node_vc_data,node_video_data,node_youtube_data,node_span_masking toneAmber
class node_starc,node_t5_model,node_vision,node_asot,node_texttitling,node_saliency_loss toneMint
class node_dvc_eval,node_vc_eval,node_soda_entry,node_soda,node_anet_data,node_cider,node_cider_scorer,node_meteor toneRose
```

## Components

### Training and inference

- [Training runner](https://github.com/ermitaju1/starc/blob/main/dvc_ret.py): `dvc_ret.py`
- [Run configuration](https://github.com/ermitaju1/starc/blob/main/args.py): `args.py`

### Video data

- [Chapter conversion](https://github.com/ermitaju1/starc/blob/main/collection/desc2chapters.py): `collection/desc2chapters.py`
- [Chapter utilities](https://github.com/ermitaju1/starc/blob/main/collection/chapter_utils.py): `collection/chapter_utils.py`
- [Dense-caption data](https://github.com/ermitaju1/starc/blob/main/dataset/dvc_dataset.py): `dataset/dvc_dataset.py`
- [Alternate DVC data](https://github.com/ermitaju1/starc/blob/main/dataset/dvc_dataset_2.py): `dataset/dvc_dataset_2.py`
- [Video-caption data](https://github.com/ermitaju1/starc/blob/main/dataset/vc_dataset.py): `dataset/vc_dataset.py`
- [Video features](https://github.com/ermitaju1/starc/blob/main/dataset/video_dataset.py): `dataset/video_dataset.py`
- [YouTube data](https://github.com/ermitaju1/starc/blob/main/dataset/yt_dataset.py): `dataset/yt_dataset.py`
- [T5 span masking](https://github.com/ermitaju1/starc/blob/main/util/t5.py): `util/t5.py`

### Captioning model

- [STaRC model](https://github.com/ermitaju1/starc/blob/main/model/STaRC.py): `model/STaRC.py`
- [T5 generation](https://github.com/ermitaju1/starc/blob/main/model/modeling_t5.py): `model/modeling_t5.py`
- [Vision encoder](https://github.com/ermitaju1/starc/blob/main/model/vit.py): `model/vit.py`
- [ASOT segmentation](https://github.com/ermitaju1/starc/blob/main/model/asot.py): `model/asot.py`
- [Text titling](https://github.com/ermitaju1/starc/blob/main/model/texttitling.py): `model/texttitling.py`
- [Saliency loss](https://github.com/ermitaju1/starc/blob/main/util/HD_loss.py): `util/HD_loss.py`

### Evaluation

- [DVC evaluator](https://github.com/ermitaju1/starc/blob/main/dvc_eval/eval_dvc.py): `dvc_eval/eval_dvc.py`
- [VC evaluator](https://github.com/ermitaju1/starc/blob/main/dvc_eval/eval_vc.py): `dvc_eval/eval_vc.py`
- [SODA evaluation entry](https://github.com/ermitaju1/starc/blob/main/dvc_eval/eval_soda.py): `dvc_eval/eval_soda.py`
- [SODA scoring](https://github.com/ermitaju1/starc/blob/main/dvc_eval/SODA/soda.py): `dvc_eval/SODA/soda.py`
- [Caption evaluation data](https://github.com/ermitaju1/starc/blob/main/dvc_eval/SODA/dataset.py): `dvc_eval/SODA/dataset.py`
- [CIDEr metric](https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/cider/cider.py): `dvc_eval/pycocoevalcap/cider/cider.py`
- [CIDEr scorer](https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/cider/cider_scorer.py): `dvc_eval/pycocoevalcap/cider/cider_scorer.py`
- [METEOR metric](https://github.com/ermitaju1/starc/blob/main/dvc_eval/pycocoevalcap/meteor/meteor.py): `dvc_eval/pycocoevalcap/meteor/meteor.py`

### Other components

- **Research user**

## Connections

- Research user → Training runner: launches (evidence: `dvc_ret.py`)
- Training runner → Run configuration: reads options (evidence: `dvc_ret.py`)
- Training runner → STaRC model: runs model (evidence: `dvc_ret.py`)
- Training runner → DVC evaluator: evaluates captions (evidence: `dvc_ret.py`)
- Training runner → SODA evaluation entry: evaluates SODA (evidence: `dvc_ret.py`)
- Chapter conversion → Chapter utilities: uses (evidence: `collection/desc2chapters.py`)
- Dense-caption data → T5 span masking: uses (evidence: `dataset/dvc_dataset.py`)
- Alternate DVC data → T5 span masking: uses (evidence: `dataset/dvc_dataset_2.py`)
- YouTube data → T5 span masking: uses (evidence: `dataset/yt_dataset.py`)
- STaRC model → T5 generation: generates with (evidence: `model/STaRC.py`)
- STaRC model → Vision encoder: encodes video (evidence: `model/STaRC.py`)
- STaRC model → ASOT segmentation: segments video (evidence: `model/STaRC.py`)
- STaRC model → Saliency loss: uses loss (evidence: `model/STaRC.py`)
- SODA evaluation entry → Caption evaluation data: loads data (evidence: `dvc_eval/eval_soda.py`)
- SODA evaluation entry → SODA scoring: runs scorer (evidence: `dvc_eval/eval_soda.py`)
- SODA scoring → Caption evaluation data: scores data (evidence: `dvc_eval/SODA/soda.py`)
- SODA scoring → CIDEr metric: uses metric (evidence: `dvc_eval/SODA/soda.py`)
- SODA scoring → METEOR metric: uses metric (evidence: `dvc_eval/SODA/soda.py`)
- DVC evaluator → CIDEr metric: uses metric (evidence: `dvc_eval/eval_dvc.py`)
- DVC evaluator → METEOR metric: uses metric (evidence: `dvc_eval/eval_dvc.py`)
- VC evaluator → CIDEr metric: uses metric (evidence: `dvc_eval/eval_vc.py`)
- VC evaluator → METEOR metric: uses metric (evidence: `dvc_eval/eval_vc.py`)
- CIDEr metric → CIDEr scorer: delegates scoring (evidence: `dvc_eval/pycocoevalcap/cider/cider.py`)
