# llsalazarll/magemotionssj architecture

> Architecture diagram of the GitHub repository llsalazarll/magemotionssj, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/llsalazarll/magemotionssj
- Repository: https://github.com/llsalazarll/magemotionssj

## Overview

Mage_Motion_SSJ is a real-time hand-gesture racing controller. In the directly shown `main.py` workflow, a webcam frame is processed by hand tracking; detected hand landmarks drive manual steering, throttle and braking, filters smooth inputs, and a virtual gamepad sends controls to a game. The README also documents a separate temporal-model assist path and a behavior-cloning training workflow. The AI inference implementation and controller implementation were not excerpted, so those links rely on README statements rather than inspected calls.

Main entry points: [`main.py`](https://github.com/llsalazarll/magemotionssj/blob/main/main.py) (Manual Runtime), [`realtime/ai_inference.py`](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py) (Temporal Inference), [`realtime/gamepad_controller.py`](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/gamepad_controller.py) (Gamepad Controller), [`models/gesture_driver_gru.keras`](https://github.com/llsalazarll/magemotionssj/blob/main/models/gesture_driver_gru.keras) (LSTM / GRU Models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Hand Capture"]
  node_tracker["Hand Tracker<br/>[hand_tracker.py]"]
  node_normalizer["Landmark Normalizer<br/>[landmark_utils.py]"]
end

subgraph group_control["Gesture Control"]
  node_main["Manual Runtime<br/>[main.py]"]
  node_gesturemap["Gesture Mapping<br/>[main.py]"]
  node_filters["EMA Filters<br/>[smoothing.py]"]
  node_overlay["Control Display<br/>[main.py]"]
  node_controller["Gamepad Controller"]
  node_gamepad["Virtual Xbox Pad"]
end

subgraph group_ai["AI Assist"]
  node_aiinference["Temporal Inference<br/>[ai_inference.py]"]
  node_sequence["Sequence Buffer<br/>[ai_inference.py]"]
  node_hybrid["Hybrid Assist<br/>[ai_inference.py]"]
  node_stabilizer["Steering Stabilizer<br/>[ai_inference.py]"]
  node_models[("LSTM / GRU Models")]
end

subgraph group_training["Model Training"]
  node_sequencer["Sequence Generator"]
  node_lstmtrainer["LSTM Trainer<br/>[train_lstm.py]"]
  node_grutrainer["GRU Trainer<br/>[train_gru.py]"]
end

node_driver(("Driver"))
node_webcam["Webcam"]
node_game["Controller-Supported Game"]

node_driver -->|"provides video"| node_webcam
node_webcam -->|"supplies frames"| node_main
node_main -->|"processes frames"| node_tracker
node_tracker -->|"normalizes landmarks"| node_normalizer
node_main -->|"maps gestures"| node_gesturemap
node_gesturemap -->|"smooths controls"| node_filters
node_main -->|"updates controls"| node_controller
node_main -->|"renders controls"| node_overlay
node_driver -->|"drives gestures"| node_aiinference
node_aiinference -->|"buffers frames"| node_sequence
node_aiinference -->|"predicts with"| node_models
node_aiinference -->|"assists steering"| node_hybrid
node_hybrid -->|"stabilizes steering"| node_stabilizer
node_aiinference -->|"sends controls"| node_controller
node_controller -->|"emulates input"| node_gamepad
node_gamepad -->|"controls"| node_game
node_sequencer -->|"prepares sequences"| node_lstmtrainer
node_sequencer -->|"prepares sequences"| node_grutrainer
node_lstmtrainer -->|"trains LSTM"| node_models
node_grutrainer -->|"trains GRU"| node_models

click node_main "https://github.com/llsalazarll/magemotionssj/blob/main/main.py"
click node_tracker "https://github.com/llsalazarll/magemotionssj/blob/main/mediapipe_utils/hand_tracker.py"
click node_normalizer "https://github.com/llsalazarll/magemotionssj/blob/main/mediapipe_utils/landmark_utils.py"
click node_gesturemap "https://github.com/llsalazarll/magemotionssj/blob/main/main.py"
click node_filters "https://github.com/llsalazarll/magemotionssj/blob/main/realtime/smoothing.py"
click node_overlay "https://github.com/llsalazarll/magemotionssj/blob/main/main.py"
click node_aiinference "https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py"
click node_sequence "https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py"
click node_hybrid "https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py"
click node_stabilizer "https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py"
click node_models "https://github.com/llsalazarll/magemotionssj/blob/main/models/gesture_driver_gru.keras"
click node_controller "https://github.com/llsalazarll/magemotionssj/blob/main/realtime/gamepad_controller.py"
click node_sequencer "https://github.com/llsalazarll/magemotionssj/blob/main/training/sequence_generator.py"
click node_lstmtrainer "https://github.com/llsalazarll/magemotionssj/blob/main/training/train_lstm.py"
click node_grutrainer "https://github.com/llsalazarll/magemotionssj/blob/main/training/train_gru.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tracker,node_normalizer toneBlue
class node_main,node_gesturemap,node_filters,node_overlay,node_controller,node_gamepad toneAmber
class node_aiinference,node_sequence,node_hybrid,node_stabilizer,node_models toneMint
class node_sequencer,node_lstmtrainer,node_grutrainer toneRose
class node_driver,node_webcam,node_game toneIndigo
```

## Components

### Hand Capture

- [Hand Tracker](https://github.com/llsalazarll/magemotionssj/blob/main/mediapipe_utils/hand_tracker.py): `mediapipe_utils/hand_tracker.py`
- [Landmark Normalizer](https://github.com/llsalazarll/magemotionssj/blob/main/mediapipe_utils/landmark_utils.py): `mediapipe_utils/landmark_utils.py`

### Gesture Control

- [Manual Runtime](https://github.com/llsalazarll/magemotionssj/blob/main/main.py): `main.py`
- [Gesture Mapping](https://github.com/llsalazarll/magemotionssj/blob/main/main.py): `main.py`
- [EMA Filters](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/smoothing.py): `realtime/smoothing.py`
- [Control Display](https://github.com/llsalazarll/magemotionssj/blob/main/main.py): `main.py`
- [Gamepad Controller](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/gamepad_controller.py): `realtime/gamepad_controller.py`
- **Virtual Xbox Pad**

### AI Assist

- [Temporal Inference](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py): `realtime/ai_inference.py`
- [Sequence Buffer](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py): `realtime/ai_inference.py`
- [Hybrid Assist](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py): `realtime/ai_inference.py`
- [Steering Stabilizer](https://github.com/llsalazarll/magemotionssj/blob/main/realtime/ai_inference.py): `realtime/ai_inference.py`
- [LSTM / GRU Models](https://github.com/llsalazarll/magemotionssj/blob/main/models/gesture_driver_gru.keras): `models/gesture_driver_gru.keras`

### Model Training

- [Sequence Generator](https://github.com/llsalazarll/magemotionssj/blob/main/training/sequence_generator.py): `training/sequence_generator.py`
- [LSTM Trainer](https://github.com/llsalazarll/magemotionssj/blob/main/training/train_lstm.py): `training/train_lstm.py`
- [GRU Trainer](https://github.com/llsalazarll/magemotionssj/blob/main/training/train_gru.py): `training/train_gru.py`

### Other components

- **Driver**
- **Webcam**
- **Controller-Supported Game**

## Connections

- Driver → Webcam: provides video (evidence: `readme.md`)
- Webcam → Manual Runtime: supplies frames (evidence: `main.py`)
- Manual Runtime → Hand Tracker: processes frames (evidence: `main.py`)
- Hand Tracker → Landmark Normalizer: normalizes landmarks (evidence: `mediapipe_utils/hand_tracker.py`)
- Manual Runtime → Gesture Mapping: maps gestures (evidence: `main.py`)
- Gesture Mapping → EMA Filters: smooths controls (evidence: `main.py`)
- Manual Runtime → Gamepad Controller: updates controls (evidence: `main.py`)
- Manual Runtime → Control Display: renders controls (evidence: `main.py`)
- Driver → Temporal Inference: drives gestures (evidence: `readme.md`)
- Temporal Inference → Sequence Buffer: buffers frames (evidence: `readme.md`)
- Temporal Inference → LSTM / GRU Models: predicts with (evidence: `readme.md`)
- Temporal Inference → Hybrid Assist: assists steering (evidence: `readme.md`)
- Hybrid Assist → Steering Stabilizer: stabilizes steering (evidence: `readme.md`)
- Temporal Inference → Gamepad Controller: sends controls (evidence: `readme.md`)
- Gamepad Controller → Virtual Xbox Pad: emulates input (evidence: `readme.md`)
- Virtual Xbox Pad → Controller-Supported Game: controls (evidence: `readme.md`)
- Sequence Generator → LSTM Trainer: prepares sequences (evidence: `readme.md`)
- Sequence Generator → GRU Trainer: prepares sequences (evidence: `readme.md`)
- LSTM Trainer → LSTM / GRU Models: trains LSTM (evidence: `training/train_lstm.py`)
- GRU Trainer → LSTM / GRU Models: trains GRU (evidence: `training/train_gru.py`)
