How vchitect/vbench works
VBench is a family of benchmarks for evaluating generated videos. The main workflow is to select videos and evaluation dimensions, build prompt/video metadata, run dimension-specific evaluators, and save scores. The repository also documents VBench-2.0 and VBench++ tracks for image-to-video, long-video, and trustworthiness evaluation. The sampled source confirms orchestration and parts of the long-video and camera-motion implementations; other dimension internals are represented only where the tree or README supports them.
Main entry points: vbench/__init__.py VBench evaluator · vbench2_beta_long/__init__.py Long-video evaluation · vbench2/__init__.py V2 evaluator · vbench2_beta_trustworthiness/__init__.py Trust evaluation
Components 21
VBench Core
- Evaluation CLI
vbench/launch/evaluate.py - VBench evaluator
vbench/__init__.py - Prompt-video metadata
vbench/__init__.py - Quality dimensions
vbench - Content dimensions
vbench - Temporal dimensions
vbench - Static filtering
vbench/cli/static_filter.py
VBench 2.0
- V2 evaluation CLI
VBench-2.0/evaluate.py - V2 evaluator
VBench-2.0/vbench2/__init__.py - V2 dimensions
VBench-2.0/vbench2 - V2 score calculation
VBench-2.0/scripts/cal_final_score.py
Image To Video
- I2V evaluation
vbench2_beta_i2v - Camera motion
vbench2_beta_i2v/camera_motion.py
Long Video
- Long-video evaluation
vbench2_beta_long/__init__.py - Clip and scene preparation
vbench2_beta_long/utils.py - Long-video dimensions
vbench2_beta_long
Trustworthiness
- Trust evaluation
vbench2_beta_trustworthiness/__init__.py - Fairness and safety
vbench2_beta_trustworthiness
Other components
- Benchmark user
- Generated videos
- Evaluation results
Connections 24
- Benchmark user → Evaluation CLI: selects run
- Generated videos → VBench evaluator: provides videos
- Evaluation CLI → VBench evaluator: starts evaluation
- VBench evaluator → Prompt-video metadata: builds metadata
- VBench evaluator → Quality dimensions: dispatches dimensions
- VBench evaluator → Content dimensions: dispatches dimensions
- VBench evaluator → Temporal dimensions: dispatches dimensions
- VBench evaluator → Evaluation results: saves scores
- Generated videos → Static filtering: filters videos
- Benchmark user → V2 evaluation CLI: selects run
- Generated videos → V2 evaluator: provides videos
- V2 evaluation CLI → V2 evaluator: starts evaluation
- V2 evaluator → V2 dimensions: dispatches dimensions
- V2 evaluator → Evaluation results: saves results
- Evaluation results → V2 score calculation: calculates final scores
- Generated videos → I2V evaluation: provides videos
- I2V evaluation → Camera motion: evaluates camera motion
- Generated videos → Long-video evaluation: provides videos
- Long-video evaluation → Clip and scene preparation: splits into clips
- Long-video evaluation → Long-video dimensions: dispatches dimensions
- Long-video evaluation → Evaluation results: saves results
- Generated videos → Trust evaluation: provides videos
- Trust evaluation → Fairness and safety: dispatches dimensions
- Trust evaluation → Evaluation results: saves results