How rsankar1996/gesture-detection-ek-ra8p1 works
This is an embedded gesture-detection application for the RA8P1 platform. An image-ready event wakes the AI thread, which clears detection state, invokes the palm-detection API, and returns results for downstream display handling. FreeRTOS supplies scheduling and synchronization; TensorFlow Lite Micro provides the inference runtime and kernels, with optional NPU support. The repository also embeds reusable CMSIS-DSP and D/AVE2D graphics libraries. Several application integrations are unsampled, so those relationships are marked optional or omitted.
Components 16
Application Flow
- AI Thread
src/ai_inference_thread_entry.c - Model Input Buffer
src/ai_inference_thread_entry.c - Detection Results
src/ai_inference_thread_entry.c - Palm Detection API
src/ai_application/common/Main.cc - Palm Detection Loop
src/ai_application/common
RTOS Runtime
- FreeRTOS Scheduler
ra/aws/FreeRTOS/FreeRTOS/Source/tasks.c
AI Inference
- TFLite Micro Runtime
ra/npu/tflite-micro/tensorflow/lite/micro - Embedding Kernel
ra/npu/tflite-micro/tensorflow/lite/micro/kernels/embedding_lookup.cc - Micro Memory API
ra/npu/tflite-micro/tensorflow/lite/micro/micro_context.h - NPU Support
ra/npu
DSP and Graphics
- CMSIS-DSP Library
ra/arm/CMSIS-DSP - Control Transforms
ra/arm/CMSIS-DSP/Include/dsp/controller_functions.h - Weighted Statistics
ra/arm/CMSIS-DSP/Source/SupportFunctions/arm_weighted_average_f32.c - 2D Rendering
ra/tes/dave2d/src/dave_context.c
Other components
- Image Capture
- MIPI Display
Connections 14
- Image Capture → AI Thread: provides frames
- AI Thread → Model Input Buffer: uses input
- AI Thread → FreeRTOS Scheduler: waits and delays
- AI Thread → Detection Results: clears results
- AI Thread → Palm Detection API: calls inference
- Palm Detection API → Palm Detection Loop: delegates detection
- Palm Detection Loop → TFLite Micro Runtime: runs model
- TFLite Micro Runtime → Embedding Kernel: dispatches kernel
- Embedding Kernel → Micro Memory API: allocates tensors
- TFLite Micro Runtime → NPU Support: uses accelerator
- Detection Results → MIPI Display: supplies detections
- MIPI Display → 2D Rendering: renders overlay
- CMSIS-DSP Library → Control Transforms: exposes transforms
- CMSIS-DSP Library → Weighted Statistics: exposes statistics