How ahmetgunduz/real-time-gesrec works
This is a PyTorch research application for offline and real-time dynamic hand-gesture recognition. The principal workflow samples video frames, applies spatial and temporal transforms, and feeds 3D CNNs to classify gestures; online evaluation adds sliding-window detection and one-time gesture activation. Training and validation provide an offline model-development workflow. The graph includes dataset preparation and documented evaluation capabilities; several subsystem links rely on the README or SOURCE INDEX because their implementation was not excerpted.
Main entry points: main.py Training entry point · dataset.py Dataset selection · online_test_video.py Online video test · model.py Model factory
Components 26
Video data
- Dataset preparation
utils/ego_prepare.py - Gesture annotations
utils/egogesture_json.py - Dataset selection
dataset.py - EgoGesture clips
datasets/egogesture.py - EgoGesture windows
datasets/egogesture_online.py - NVGesture clips
datasets/nv.py - NVGesture windows
datasets/nv_online.py - Other datasets
datasets
Recognition runtime
- Clip transforms
spatial_transforms.py - Temporal sampling
temporal_transforms.py - Class targets
target_transforms.py - Offline inference
inference.py - Online video test
online_test_video.py - Online clip test
online_test_wo_detector.py - Detection metrics
utils.py
3D CNN models
- Model factory
model.py - 3D ResNet
models/resnet.py - ShuffleNetV2
models/shufflenetv2.py - CNN architectures
models
Training and evaluation
- Run configuration
opts.py - Model training
train.py - Validation
validation.py - Training entry point
main.py - Dataset evaluation
utils/eval_kinetics.py
Other components
- Researcher
- Video datasets
Connections 35
- Researcher → Training entry point: configures training
- Researcher → Online video test: runs online test
- Video datasets → Dataset preparation: provides source data
- Dataset preparation → Gesture annotations: prepares frame lists
- Gesture annotations → Dataset selection: supplies annotations
- Training entry point → Run configuration: parses options
main.py - Training entry point → Dataset selection: builds datasets
main.py - Dataset selection → EgoGesture clips: selects dataset
dataset.py - Dataset selection → NVGesture clips: selects dataset
dataset.py - Dataset selection → EgoGesture windows: selects online data
dataset.py - Dataset selection → NVGesture windows: selects online data
dataset.py - Dataset selection → Other datasets: selects dataset
dataset.py - EgoGesture clips → Clip transforms: uses spatial transforms
datasets/egogesture.py - EgoGesture windows → Clip transforms: uses spatial transforms
datasets/egogesture_online.py - EgoGesture windows → Temporal sampling: uses temporal sampling
datasets/egogesture_online.py - NVGesture clips → Clip transforms: uses spatial transforms
datasets/nv.py - NVGesture windows → Clip transforms: uses spatial transforms
datasets/nv_online.py - NVGesture windows → Temporal sampling: uses temporal sampling
datasets/nv_online.py - Training entry point → Clip transforms: configures transforms
main.py - Training entry point → Temporal sampling: samples training clips
main.py - Training entry point → Class targets: creates class targets
main.py - Training entry point → Model factory: creates model
main.py - Model factory → 3D ResNet: constructs architecture
model.py - Model factory → ShuffleNetV2: constructs architecture
model.py - Model factory → CNN architectures: constructs architecture
model.py - Training entry point → Model training: runs epochs
main.py - Training entry point → Validation: validates model
main.py - Online video test → Run configuration: parses options
online_test_video.py - Online video test → Model factory: loads classifier
online_test_video.py - Online video test → Detection metrics: uses detection metrics
online_test_video.py - Online clip test → Model factory: loads classifier
online_test_wo_detector.py - Online video test → EgoGesture windows: reads gesture windows
- Online video test → NVGesture windows: reads gesture windows
- Online clip test → EgoGesture windows: reads gesture windows
- Dataset evaluation → Video datasets: evaluates dataset