# aaravrawal52/cg4002_b08 architecture

> Architecture diagram of the GitHub repository aaravrawal52/cg4002_b08, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aaravrawal52/cg4002_b08
- Repository: https://github.com/aaravrawal52/cg4002_b08

## Overview

This repository implements a dual-glove gesture-recognition system: glove sensor samples travel over BLE to a Mac gateway, then over a TCP/SSH tunnel to an Ultra96 for gesture classification; results return to the gateway and reach a phone visualiser over WebSocket. Separate AI modules support data preparation and model training/deployment. The README documents the end-to-end links; sampled source shows sensor acquisition and inference details, but not the gateway implementation or Unity receiver internals.

Main entry points: [`comms/ultra96/ultra96_server.py`](https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/ultra96/ultra96_server.py) (Ultra96 server), [`comms/laptop/ble_gateway.py`](https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/laptop/ble_gateway.py) (Mac BLE gateway), [`ai/deploy/inference.py`](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/deploy/inference.py) (DPU classifier), [`Firebeetle/Flex Sensor Ver/sketch_nov5/SensorManager.cpp`](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/SensorManager.cpp) (Flex sensors)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_gloves["Glove sensing"]
  node_flex_sensor["Flex sensors<br/>[SensorManager.cpp]"]
  node_hall_sensor["Hall sensors<br/>[SensorManager.cpp]"]
  node_flex_processor["Flex gesture processor"]
  node_hall_processor["Hall gesture processor"]
  node_flex_types["Flex sample types<br/>[SensorTypes.h]"]
  node_hall_types["Hall sample types<br/>[SensorTypes.h]"]
end

subgraph group_comms["Communications"]
  node_gateway["Mac BLE gateway<br/>[ble_gateway.py]"]
  node_ultra_server["Ultra96 server<br/>[ultra96_server.py]"]
end

subgraph group_inference["Gesture inference"]
  node_server_ingest["Session ingestion<br/>[data_ingestion.py]"]
  node_segmenter["Gesture segmenter<br/>[data_ingestion.py]"]
  node_classifier["DPU classifier<br/>[inference.py]"]
  node_resampler["Window resampler<br/>[resample.py]"]
  node_deployment[("Deployed model artifacts")]
end

subgraph group_ml["AI development"]
  node_collection["Sensor data collection"]
  node_preprocessing["Data preprocessing"]
  node_training["Model training<br/>[train.py]"]
  node_model["CNN model<br/>[model.py]"]
  node_quantization["Vitis quantization<br/>[quantize_model.py]"]
end

subgraph group_visualiser["Visualisation"]
  node_phone["Phone visualiser"]
end

node_user(("Glove wearer"))

node_user -->|"moves hand"| node_flex_sensor
node_user -->|"moves hand"| node_hall_sensor
node_flex_sensor -.->|"supplies samples"| node_flex_processor
node_hall_sensor -.->|"supplies samples"| node_hall_processor
node_flex_sensor -->|"fills sample"| node_flex_types
node_hall_sensor -->|"fills sample"| node_hall_types
node_flex_processor -->|"sends BLE data"| node_gateway
node_hall_processor -->|"sends BLE data"| node_gateway
node_gateway -->|"forwards TCP data"| node_ultra_server
node_ultra_server -->|"returns gestures"| node_gateway
node_gateway -->|"pushes WebSocket results"| node_phone
node_ultra_server -->|"parses packets"| node_server_ingest
node_ultra_server -->|"segments sessions"| node_segmenter
node_ultra_server -->|"classifies segments"| node_classifier
node_classifier -->|"segments live rows"| node_segmenter
node_classifier -->|"resamples windows"| node_resampler
node_collection -.->|"prepares training data"| node_preprocessing
node_training -->|"builds network"| node_model
node_quantization -->|"loads network"| node_model
node_quantization -->|"exports XModel"| node_deployment
node_classifier -->|"loads model artifacts"| node_deployment

click node_flex_sensor "https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/SensorManager.cpp"
click node_hall_sensor "https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Hall%20Sensor%20Ver/sketch_nov5/SensorManager.cpp"
click node_flex_processor "https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/GestureProcessor.cpp"
click node_hall_processor "https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Hall%20Sensor%20Ver/sketch_nov5/GestureProcessor.cpp"
click node_flex_types "https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/SensorTypes.h"
click node_hall_types "https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Hall%20Sensor%20Ver/sketch_nov5/SensorTypes.h"
click node_gateway "https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/laptop/ble_gateway.py"
click node_phone "https://github.com/aaravrawal52/cg4002_b08/tree/main/visualiser/v3"
click node_ultra_server "https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/ultra96/ultra96_server.py"
click node_server_ingest "https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/ultra96/data_ingestion.py"
click node_segmenter "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/data_ingestion.py"
click node_classifier "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/deploy/inference.py"
click node_resampler "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/deploy/resample.py"
click node_collection "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/sensorcollection.py"
click node_preprocessing "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/data_preprocessing.py"
click node_training "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/train.py"
click node_model "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/model.py"
click node_quantization "https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/vitis_ai/quantize_model.py"
click node_deployment "https://github.com/aaravrawal52/cg4002_b08/tree/main/ai/deploy"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_flex_sensor,node_hall_sensor,node_flex_processor,node_hall_processor,node_flex_types,node_hall_types toneBlue
class node_gateway,node_ultra_server toneAmber
class node_server_ingest,node_segmenter,node_classifier,node_resampler,node_deployment toneMint
class node_collection,node_preprocessing,node_training,node_model,node_quantization toneRose
class node_phone,node_user toneIndigo
```

## Components

### Glove sensing

- [Flex sensors](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/SensorManager.cpp): `Firebeetle/Flex Sensor Ver/sketch_nov5/SensorManager.cpp`
- [Hall sensors](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Hall%20Sensor%20Ver/sketch_nov5/SensorManager.cpp): `Firebeetle/Hall Sensor Ver/sketch_nov5/SensorManager.cpp`
- [Flex gesture processor](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/GestureProcessor.cpp): `Firebeetle/Flex Sensor Ver/sketch_nov5/GestureProcessor.cpp`
- [Hall gesture processor](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Hall%20Sensor%20Ver/sketch_nov5/GestureProcessor.cpp): `Firebeetle/Hall Sensor Ver/sketch_nov5/GestureProcessor.cpp`
- [Flex sample types](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Flex%20Sensor%20Ver/sketch_nov5/SensorTypes.h): `Firebeetle/Flex Sensor Ver/sketch_nov5/SensorTypes.h`
- [Hall sample types](https://github.com/aaravrawal52/cg4002_b08/blob/main/Firebeetle/Hall%20Sensor%20Ver/sketch_nov5/SensorTypes.h): `Firebeetle/Hall Sensor Ver/sketch_nov5/SensorTypes.h`

### Communications

- [Mac BLE gateway](https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/laptop/ble_gateway.py): `comms/laptop/ble_gateway.py`
- [Ultra96 server](https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/ultra96/ultra96_server.py): `comms/ultra96/ultra96_server.py`

### Gesture inference

- [Session ingestion](https://github.com/aaravrawal52/cg4002_b08/blob/main/comms/ultra96/data_ingestion.py): `comms/ultra96/data_ingestion.py`
- [Gesture segmenter](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/data_ingestion.py): `ai/data_ingestion.py`
- [DPU classifier](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/deploy/inference.py): `ai/deploy/inference.py`
- [Window resampler](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/deploy/resample.py): `ai/deploy/resample.py`
- [Deployed model artifacts](https://github.com/aaravrawal52/cg4002_b08/tree/main/ai/deploy): `ai/deploy`

### AI development

- [Sensor data collection](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/sensorcollection.py): `ai/sensorcollection.py`
- [Data preprocessing](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/data_preprocessing.py): `ai/data_preprocessing.py`
- [Model training](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/train.py): `ai/train.py`
- [CNN model](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/model.py): `ai/model.py`
- [Vitis quantization](https://github.com/aaravrawal52/cg4002_b08/blob/main/ai/vitis_ai/quantize_model.py): `ai/vitis_ai/quantize_model.py`

### Visualisation

- [Phone visualiser](https://github.com/aaravrawal52/cg4002_b08/tree/main/visualiser/v3): `visualiser/v3`

### Other components

- **Glove wearer**

## Connections

- Glove wearer → Flex sensors: moves hand (evidence: `README.md`)
- Glove wearer → Hall sensors: moves hand (evidence: `README.md`)
- Flex sensors → Flex gesture processor: supplies samples
- Hall sensors → Hall gesture processor: supplies samples
- Flex sensors → Flex sample types: fills sample (evidence: `Firebeetle/Flex Sensor Ver/sketch_nov5/SensorManager.cpp`)
- Hall sensors → Hall sample types: fills sample (evidence: `Firebeetle/Hall Sensor Ver/sketch_nov5/SensorManager.cpp`)
- Flex gesture processor → Mac BLE gateway: sends BLE data (evidence: `README.md`)
- Hall gesture processor → Mac BLE gateway: sends BLE data (evidence: `README.md`)
- Mac BLE gateway → Ultra96 server: forwards TCP data (evidence: `README.md`)
- Ultra96 server → Mac BLE gateway: returns gestures (evidence: `README.md`)
- Mac BLE gateway → Phone visualiser: pushes WebSocket results (evidence: `README.md`)
- Ultra96 server → Session ingestion: parses packets (evidence: `comms/ultra96/ultra96_server.py`)
- Ultra96 server → Gesture segmenter: segments sessions (evidence: `comms/ultra96/ultra96_server.py`)
- Ultra96 server → DPU classifier: classifies segments (evidence: `comms/ultra96/ultra96_server.py`)
- DPU classifier → Gesture segmenter: segments live rows (evidence: `ai/deploy/inference.py`)
- DPU classifier → Window resampler: resamples windows (evidence: `ai/deploy/inference.py`)
- Sensor data collection → Data preprocessing: prepares training data
- Model training → CNN model: builds network (evidence: `ai/train.py`)
- Vitis quantization → CNN model: loads network (evidence: `ai/vitis_ai/quantize_model.py`)
- Vitis quantization → Deployed model artifacts: exports XModel (evidence: `ai/vitis_ai/quantize_model.py`)
- DPU classifier → Deployed model artifacts: loads model artifacts (evidence: `ai/deploy/inference.py`)
