How uzh-rpg/rvt works
RVT trains and evaluates recurrent object detectors from Gen1/Gen4 event-camera sequences: data loading and event representation feed a stateful MaxViT-RNN/YOLOX detector, whose predictions are measured with Prophesee metrics and optionally visualized/logged. The README also documents dataset preprocessing and streamed or random sampling. The graph follows documented workflows and sampled call sites; several internal dataset-to-loader and detector-to-head links are inferred only where module responsibilities are explicit, so unsampled wiring is omitted.
Main entry points: genx.py GenX data module · detection.py Detection module · dataset_streaming.py Stream sampling · dataset_rnd.py Random sampling
Components 21
Event data
- Dataset preprocessing
scripts/genx/preprocess_dataset.py - GenX data module
modules/data/genx.py - Sequence datasets
data/genx_utils/sequence_base.py - Stream sampling
data/genx_utils/dataset_streaming.py - Random sampling
data/genx_utils/dataset_rnd.py - Batch collation
data/genx_utils/collate.py - Event representation
data/utils/representations.py - Spatial augmentation
data/utils/augmentor.py - Object labels
data/genx_utils/labels.py
Detection model
- MaxViT-RNN backbone
models/detection/recurrent_backbone/maxvit_rnn.py - YOLOX detector
models/detection/yolox_extension/models/detector.py - YOLOX head
models/detection/yolox/models/yolo_head.py
Training and evaluation
- Training entry
train.py - Validation entry
validation.py - Detection module
modules/detection.py - Recurrent state helpers
modules/utils/detection.py
Metrics and logging
- Prophesee evaluation
utils/evaluation/prophesee/evaluator.py - Detection callbacks
callbacks/detection.py - W&B logger
loggers/wandb_logger.py
Other components
- Researcher
- Weights & Biases
Connections 28
- Researcher → Dataset preprocessing: prepares data
- Researcher → Training entry: starts training
- Researcher → Validation entry: starts evaluation
- Dataset preprocessing → Sequence datasets: creates dataset files
- Training entry → GenX data module: configures loading
- Validation entry → GenX data module: configures loading
- GenX data module → Stream sampling: builds stream dataset
- GenX data module → Random sampling: builds random dataset
- Stream sampling → Sequence datasets: reads sequences
- Random sampling → Sequence datasets: reads sequences
- GenX data module → Batch collation: selects collator
- Stream sampling → Event representation: forms event tensors
- Random sampling → Event representation: forms event tensors
- Stream sampling → Object labels: loads labels
- Random sampling → Object labels: loads labels
- Spatial augmentation → Object labels: transforms boxes
- GenX data module → Training entry: provides batches
- GenX data module → Validation entry: provides batches
- Training entry → Detection module: trains module
- Validation entry → Detection module: evaluates module
- Detection module → YOLOX detector: runs detector
- Detection module → Recurrent state helpers: manages recurrent state
- YOLOX detector → MaxViT-RNN backbone: extracts features
- YOLOX detector → YOLOX head: predicts boxes
- Detection module → Prophesee evaluation: submits predictions
- Detection module → Detection callbacks: emits visualization hooks
- Detection callbacks → W&B logger: logs visualizations
- W&B logger → Weights & Biases: uploads metrics