How cvlab-kaist/locotrack works
LocoTrack is a point-tracking research product with JAX and PyTorch implementations, training and evaluation support, and an interactive video demo. The principal user workflow is to provide a video and query points, run the tracker, and view a rendered track video. The graph also captures the training-data and evaluation branches documented in the README. Some implementation excerpts are partial, so uncertain internal wiring is omitted.
Main entry points: demo.py Gradio demo · locotrack_model.py PyTorch tracker · experiment.py PyTorch training · supervised_point_prediction.py JAX prediction task
Components 17
Interactive demo
- Video and queries
- Gradio demo
demo/demo.py - Rendered tracks
Tracking models
- PyTorch tracker
locotrack_pytorch/models/locotrack_model.py - Feature networks
locotrack_pytorch/models/nets.py - JAX tracker
locotrack/locotrack_model.py - JAX ResNet
locotrack/models/resnet.py
Training and evaluation
- PyTorch training
locotrack_pytorch/experiment.py - JAX prediction task
locotrack/supervised_point_prediction.py
Dataset handling
- PyTorch evaluation data
locotrack_pytorch/data/evaluation_datasets.py - PyTorch Kubric pipeline
locotrack_pytorch/data/kubric_data.py - JAX Kubric pipeline
locotrack/data/kubric_data.py - TAP-Vid preparation
locotrack/data/generate_tapvid.py
Other components
- User
- TAP-Vid data
- Kubric training data
- Evaluation datasets
Connections 17
- User → Gradio demo: submits video and points
- Gradio demo → Video and queries: reads input
- Gradio demo → PyTorch tracker: loads and runs
- PyTorch tracker → Feature networks: uses networks
- PyTorch tracker → Rendered tracks: predicts tracks
- Gradio demo → Rendered tracks: renders output
- PyTorch training → PyTorch tracker: trains or evaluates
- PyTorch training → PyTorch Kubric pipeline: loads training data
- PyTorch training → PyTorch evaluation data: loads evaluation data
- PyTorch Kubric pipeline → Kubric training data: reads
- PyTorch evaluation data → Evaluation datasets: reads
- JAX prediction task → JAX tracker: runs prediction
- JAX tracker → JAX ResNet: uses backbone
- JAX prediction task → JAX Kubric pipeline: loads evaluation data
- JAX Kubric pipeline → Kubric training data: loads
- TAP-Vid preparation → TAP-Vid data: converts videos and tracks
- Evaluation datasets → JAX prediction task: evaluates tracker