How hanrekoen/prj371-project-sasl works
SignMasterVR is a Unity-based South African Sign Language tutor whose lesson flow runs on a Quest while gesture recognition runs on a laptop. The learner sees a tutor demonstrate a sign, attempts it, and receives feedback before the lesson advances. The laptop server tracks webcam hand landmarks, extracts features, classifies attempts, and relays results over TCP; Unity’s lesson logic decides pass/fail. The graph also includes the documented model-training and Unity-asset generation capabilities. Unity source excerpts were not provided, so those links rely on README statements; some Python runtime relationships are supported by the source index.
Main entry points: server.py Gesture server · LessonManager.cs Lesson manager · gesture_classifier.py Model validator · sasl_features.py Feature extraction
Components 19
VR lesson
- Lesson manager
SignMasterVR/Assets/Scripts/Lessons/LessonManager.cs - Lesson levels
SignMasterVR/Assets/Data/Levels - Gesture content
SignMasterVR/Assets/Data/Gestures - Tutor demo
SignMasterVR/Assets/Scripts/Tutor/TutorController.cs - Lesson UI
SignMasterVR/Assets/Scripts/UI/UIManager.cs
Gesture recognition
- Hand tracker
SignMasterVR/GestureServer/hand_tracker.py - Feature extraction
SignMasterVR/GestureServer/sasl_features.py - Model validator
SignMasterVR/GestureServer/gesture_classifier.py - SASL classifier
SignMasterVR/GestureServer/sasl_classifier.py - Placeholder classifier
SignMasterVR/GestureServer/classifier.py
Model and content
- Trained model
SignMasterVR/GestureServer/sasl_model.py - Model training
SignMasterVR/GestureServer/train_gesture_model.py - Unity asset generator
SignMasterVR/GestureServer/make_unity_assets.py - Tutor hand export
SignMasterVR/GestureServer/export_ghost_hand.py
Network transport
- Network recognizer
SignMasterVR/Assets/Scripts/ML/NetworkGestureRecognizer.cs - Gesture server
SignMasterVR/GestureServer/server.py - Wire protocol
SignMasterVR/GestureServer/protocol.py
Other components
- Learner
- Laptop webcam
Connections 23
- Learner → Lesson manager: attempts signs
README.md - Lesson manager → Lesson levels: uses levels
README.md - Lesson manager → Gesture content: checks target
README.md - Lesson manager → Tutor demo: requests demo
README.md - Lesson manager → Lesson UI: updates feedback
README.md - Lesson manager → Network recognizer: sets target
README.md - Network recognizer → Gesture server: sends target
- Gesture server → Network recognizer: sends result
- Gesture server → Wire protocol: encodes messages
SignMasterVR/GestureServer/server.py - Laptop webcam → Gesture server: provides frames
README.md - Gesture server → Hand tracker: tracks hands
SignMasterVR/GestureServer/server.py - Gesture server → SASL classifier: classifies attempts
SignMasterVR/GestureServer/server.py - Gesture server → Placeholder classifier: selects fallback
SignMasterVR/GestureServer/server.py - Hand tracker → Model validator: provides landmarks
SignMasterVR/GestureServer/gesture_classifier.py - Model validator → Feature extraction: extracts features
SignMasterVR/GestureServer/gesture_classifier.py - Model validator → Trained model: loads model
- SASL classifier → Model validator: wraps validator
SignMasterVR/GestureServer/sasl_classifier.py - Model training → Feature extraction: uses features
SignMasterVR/GestureServer/train_gesture_model.py - Model training → Trained model: trains artifact
SignMasterVR/GestureServer/train_gesture_model.py - Unity asset generator → Feature extraction: uses features
SignMasterVR/GestureServer/make_unity_assets.py - Unity asset generator → Gesture content: generates assets
README.md - Tutor hand export → Feature extraction: uses features
SignMasterVR/GestureServer/export_ghost_hand.py - Tutor hand export → Tutor demo: exports animations
README.md