How llsalazarll/magemotionssj works
Mage_Motion_SSJ is a real-time hand-gesture racing controller. In the directly shown main.py workflow, a webcam frame is processed by hand tracking; detected hand landmarks drive manual steering, throttle and braking, filters smooth inputs, and a virtual gamepad sends controls to a game. The README also documents a separate temporal-model assist path and a behavior-cloning training workflow. The AI inference implementation and controller implementation were not excerpted, so those links rely on README statements rather than inspected calls.
Main entry points: main.py Manual Runtime · ai_inference.py Temporal Inference · gamepad_controller.py Gamepad Controller · gesture_driver_gru.keras LSTM / GRU Models
Components 19
Hand Capture
- Hand Tracker
mediapipe_utils/hand_tracker.py - Landmark Normalizer
mediapipe_utils/landmark_utils.py
Gesture Control
- Manual Runtime
main.py - Gesture Mapping
main.py - EMA Filters
realtime/smoothing.py - Control Display
main.py - Gamepad Controller
realtime/gamepad_controller.py - Virtual Xbox Pad
AI Assist
- Temporal Inference
realtime/ai_inference.py - Sequence Buffer
realtime/ai_inference.py - Hybrid Assist
realtime/ai_inference.py - Steering Stabilizer
realtime/ai_inference.py - LSTM / GRU Models
models/gesture_driver_gru.keras
Model Training
- Sequence Generator
training/sequence_generator.py - LSTM Trainer
training/train_lstm.py - GRU Trainer
training/train_gru.py
Other components
- Driver
- Webcam
- Controller-Supported Game
Connections 20
- Driver → Webcam: provides video
readme.md - Webcam → Manual Runtime: supplies frames
main.py - Manual Runtime → Hand Tracker: processes frames
main.py - Hand Tracker → Landmark Normalizer: normalizes landmarks
mediapipe_utils/hand_tracker.py - Manual Runtime → Gesture Mapping: maps gestures
main.py - Gesture Mapping → EMA Filters: smooths controls
main.py - Manual Runtime → Gamepad Controller: updates controls
main.py - Manual Runtime → Control Display: renders controls
main.py - Driver → Temporal Inference: drives gestures
readme.md - Temporal Inference → Sequence Buffer: buffers frames
readme.md - Temporal Inference → LSTM / GRU Models: predicts with
readme.md - Temporal Inference → Hybrid Assist: assists steering
readme.md - Hybrid Assist → Steering Stabilizer: stabilizes steering
readme.md - Temporal Inference → Gamepad Controller: sends controls
readme.md - Gamepad Controller → Virtual Xbox Pad: emulates input
readme.md - Virtual Xbox Pad → Controller-Supported Game: controls
readme.md - Sequence Generator → LSTM Trainer: prepares sequences
readme.md - Sequence Generator → GRU Trainer: prepares sequences
readme.md - LSTM Trainer → LSTM / GRU Models: trains LSTM
training/train_lstm.py - GRU Trainer → LSTM / GRU Models: trains GRU
training/train_gru.py