How ermitaju1/starc works
STaRC is a research system for retrieval-augmented dense video captioning, emphasizing supervised saliency. The principal evidenced workflow is a user launching dvc_ret.py, which configures and runs the STaRC model, then invokes captioning and SODA evaluation. Dataset loaders, caption chapter utilities, and evaluation metrics are distinct supporting branches. The index and excerpts establish model internals and evaluator dependencies, but do not show the runner calling dataset loaders; that connection is therefore omitted.
Main entry points: dvc_ret.py Training runner · STaRC.py STaRC model · soda.py SODA scoring · cider.py CIDEr metric
Components 25
Training and inference
- Training runner
dvc_ret.py - Run configuration
args.py
Video data
- Chapter conversion
collection/desc2chapters.py - Chapter utilities
collection/chapter_utils.py - Dense-caption data
dataset/dvc_dataset.py - Alternate DVC data
dataset/dvc_dataset_2.py - Video-caption data
dataset/vc_dataset.py - Video features
dataset/video_dataset.py - YouTube data
dataset/yt_dataset.py - T5 span masking
util/t5.py
Captioning model
- STaRC model
model/STaRC.py - T5 generation
model/modeling_t5.py - Vision encoder
model/vit.py - ASOT segmentation
model/asot.py - Text titling
model/texttitling.py - Saliency loss
util/HD_loss.py
Evaluation
- DVC evaluator
dvc_eval/eval_dvc.py - VC evaluator
dvc_eval/eval_vc.py - SODA evaluation entry
dvc_eval/eval_soda.py - SODA scoring
dvc_eval/SODA/soda.py - Caption evaluation data
dvc_eval/SODA/dataset.py - CIDEr metric
dvc_eval/pycocoevalcap/cider/cider.py - CIDEr scorer
dvc_eval/pycocoevalcap/cider/cider_scorer.py - METEOR metric
dvc_eval/pycocoevalcap/meteor/meteor.py
Other components
- Research user
Connections 23
- Research user → Training runner: launches
dvc_ret.py - Training runner → Run configuration: reads options
dvc_ret.py - Training runner → STaRC model: runs model
dvc_ret.py - Training runner → DVC evaluator: evaluates captions
dvc_ret.py - Training runner → SODA evaluation entry: evaluates SODA
dvc_ret.py - Chapter conversion → Chapter utilities: uses
collection/desc2chapters.py - Dense-caption data → T5 span masking: uses
dataset/dvc_dataset.py - Alternate DVC data → T5 span masking: uses
dataset/dvc_dataset_2.py - YouTube data → T5 span masking: uses
dataset/yt_dataset.py - STaRC model → T5 generation: generates with
model/STaRC.py - STaRC model → Vision encoder: encodes video
model/STaRC.py - STaRC model → ASOT segmentation: segments video
model/STaRC.py - STaRC model → Saliency loss: uses loss
model/STaRC.py - SODA evaluation entry → Caption evaluation data: loads data
dvc_eval/eval_soda.py - SODA evaluation entry → SODA scoring: runs scorer
dvc_eval/eval_soda.py - SODA scoring → Caption evaluation data: scores data
dvc_eval/SODA/soda.py - SODA scoring → CIDEr metric: uses metric
dvc_eval/SODA/soda.py - SODA scoring → METEOR metric: uses metric
dvc_eval/SODA/soda.py - DVC evaluator → CIDEr metric: uses metric
dvc_eval/eval_dvc.py - DVC evaluator → METEOR metric: uses metric
dvc_eval/eval_dvc.py - VC evaluator → CIDEr metric: uses metric
dvc_eval/eval_vc.py - VC evaluator → METEOR metric: uses metric
dvc_eval/eval_vc.py - CIDEr metric → CIDEr scorer: delegates scoring
dvc_eval/pycocoevalcap/cider/cider.py