How rhon-dev/kumpas works
KUMPAS is an offline Android Filipino Sign Language learning app. A learner practices a sign in the camera view; native frame processing extracts landmarks, classifies a sequence on-device, compares it with a gold standard, and emits corrective feedback. The app also provides learning screens and records practice history. The repository includes a separate model-development pipeline and benchmark tools. The graph reflects the documented product workflow and shown native call paths; unsampled Flutter wiring and training-stage handoffs are left unasserted where evidence is limited.
Main entry points: MainActivity.kt Flutter host · VisionEngine.kt Vision inference · ui Learning screens · CameraPreviewView.kt Camera preview
Components 18
Mobile learning
- Learning screens
app/lib/ui - Flutter host
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/MainActivity.kt
On-device recognition
- Camera preview
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/CameraPreviewView.kt - Vision inference
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/VisionEngine.kt - Gold standards
app/android/app/src/main/assets - Corrective feedback
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/FeedbackEngine.kt
Practice records
- Attempt log
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/SessionLog.kt - Session manager
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/SessionManager.kt - Practice database
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/SessionDatabase.kt
Model development
- Dataset preprocessing
training/preprocessing - Landmark augmentation
training/augmentation/augment_landmarks.py - Model training
training/models/train_cnn_lstm.py - Feedback reference
training/feedback/feedback_engine.py - TFLite export
training/tflite_export/export_tflite.py
Performance validation
- Accuracy benchmark
benchmarking/accuracy_benchmark.py - Device benchmarks
benchmarking - Benchmark history
benchmarking/benchmark_history.json
Other components
- Learner
Connections 17
- Learner → Learning screens: practices with
README.md - Learning screens → Flutter host: uses native features
- Flutter host → Camera preview: creates preview
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/MainActivity.kt - Camera preview → Vision inference: submits frames
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/CameraPreviewView.kt - Flutter host → Vision inference: routes controls
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/MainActivity.kt - Vision inference → Gold standards: loads standards
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/VisionEngine.kt - Vision inference → Corrective feedback: compares attempt
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/VisionEngine.kt - Flutter host → Attempt log: logs results
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/MainActivity.kt - Session manager → Practice database: reads and writes
app/android/app/src/main/kotlin/com/kumpas/kumpas_app/SessionManager.kt - Learner → Dataset preprocessing: provides sign data
README.md - Dataset preprocessing → Landmark augmentation: prepares landmarks
README.md - Landmark augmentation → Model training: augments examples
README.md - Model training → TFLite export: exports model
README.md - Feedback reference → Corrective feedback: guides parity
training/feedback/feedback_engine.py - TFLite export → Accuracy benchmark: evaluates model
README.md - Accuracy benchmark → Benchmark history: records results
benchmarking/accuracy_benchmark.py - Device benchmarks → Benchmark history: records measurements
benchmarking/collect_latency.py