How aaravrawal52/cg4002_b08 works
This repository implements a dual-glove gesture-recognition system: glove sensor samples travel over BLE to a Mac gateway, then over a TCP/SSH tunnel to an Ultra96 for gesture classification; results return to the gateway and reach a phone visualiser over WebSocket. Separate AI modules support data preparation and model training/deployment. The README documents the end-to-end links; sampled source shows sensor acquisition and inference details, but not the gateway implementation or Unity receiver internals.
Main entry points: ultra96_server.py Ultra96 server · ble_gateway.py Mac BLE gateway · inference.py DPU classifier · SensorManager.cpp Flex sensors
Components 20
Glove sensing
- Flex sensors
Firebeetle/Flex Sensor Ver/sketch_nov5/SensorManager.cpp - Hall sensors
Firebeetle/Hall Sensor Ver/sketch_nov5/SensorManager.cpp - Flex gesture processor
Firebeetle/Flex Sensor Ver/sketch_nov5/GestureProcessor.cpp - Hall gesture processor
Firebeetle/Hall Sensor Ver/sketch_nov5/GestureProcessor.cpp - Flex sample types
Firebeetle/Flex Sensor Ver/sketch_nov5/SensorTypes.h - Hall sample types
Firebeetle/Hall Sensor Ver/sketch_nov5/SensorTypes.h
Communications
- Mac BLE gateway
comms/laptop/ble_gateway.py - Ultra96 server
comms/ultra96/ultra96_server.py
Gesture inference
- Session ingestion
comms/ultra96/data_ingestion.py - Gesture segmenter
ai/data_ingestion.py - DPU classifier
ai/deploy/inference.py - Window resampler
ai/deploy/resample.py - Deployed model artifacts
ai/deploy
AI development
- Sensor data collection
ai/sensorcollection.py - Data preprocessing
ai/data_preprocessing.py - Model training
ai/train.py - CNN model
ai/model.py - Vitis quantization
ai/vitis_ai/quantize_model.py
Visualisation
- Phone visualiser
visualiser/v3
Other components
- Glove wearer
Connections 21
- Glove wearer → Flex sensors: moves hand
README.md - Glove wearer → Hall sensors: moves hand
README.md - Flex sensors → Flex gesture processor: supplies samples
- Hall sensors → Hall gesture processor: supplies samples
- Flex sensors → Flex sample types: fills sample
Firebeetle/Flex Sensor Ver/sketch_nov5/SensorManager.cpp - Hall sensors → Hall sample types: fills sample
Firebeetle/Hall Sensor Ver/sketch_nov5/SensorManager.cpp - Flex gesture processor → Mac BLE gateway: sends BLE data
README.md - Hall gesture processor → Mac BLE gateway: sends BLE data
README.md - Mac BLE gateway → Ultra96 server: forwards TCP data
README.md - Ultra96 server → Mac BLE gateway: returns gestures
README.md - Mac BLE gateway → Phone visualiser: pushes WebSocket results
README.md - Ultra96 server → Session ingestion: parses packets
comms/ultra96/ultra96_server.py - Ultra96 server → Gesture segmenter: segments sessions
comms/ultra96/ultra96_server.py - Ultra96 server → DPU classifier: classifies segments
comms/ultra96/ultra96_server.py - DPU classifier → Gesture segmenter: segments live rows
ai/deploy/inference.py - DPU classifier → Window resampler: resamples windows
ai/deploy/inference.py - Sensor data collection → Data preprocessing: prepares training data
- Model training → CNN model: builds network
ai/train.py - Vitis quantization → CNN model: loads network
ai/vitis_ai/quantize_model.py - Vitis quantization → Deployed model artifacts: exports XModel
ai/vitis_ai/quantize_model.py - DPU classifier → Deployed model artifacts: loads model artifacts
ai/deploy/inference.py