How hiteshshaw/aeropoint-ai works
AeroPoint AI is a Windows on-device accessibility assistant: hand gestures and offline speech are interpreted by a central runtime and translated into desktop actions. Its main path combines webcam landmarks, gesture recognition, voice transcription and intent execution; configuration supplies accessibility profiles, while the controller performs cursor, keyboard and system-volume operations. The README also documents dwell clicking, a HUD, battery throttling, dictation, app launching and profile modes. The graph follows shown calls and documented capabilities; unsampled implementation details are not inferred.
Main entry points: main.py Assistant Runtime · speech_engine.py Voice Intents · os_actions.py System Controller · whisper_listener.py Whisper Listener
Components 13
Assistant Runtime
- Assistant Runtime
main.py - HUD and Dwell
main.py - Access Profiles
config.py
Hand Interaction
- Hand Tracker
vision/hand_tracker.py - Gesture Recognition
vision/gesture_recognizer.py
Offline Voice
- Whisper Listener
voice/whisper_listener.py - Voice Intents
voice/speech_engine.py
Desktop Control
- System Controller
controller/os_actions.py
Other components
- User
- Webcam
- Microphone
- Windows Desktop
- On-Device Models
Connections 18
- User → Webcam: gestures
README.md - Webcam → Assistant Runtime: video frames
main.py - Assistant Runtime → Hand Tracker: processes frames
main.py - Hand Tracker → Assistant Runtime: returns landmarks
vision/hand_tracker.py - Assistant Runtime → Gesture Recognition: classifies landmarks
main.py - User → Microphone: speaks commands
README.md - Microphone → Whisper Listener: audio stream
voice/whisper_listener.py - Whisper Listener → On-Device Models: transcribes with
voice/whisper_listener.py - Whisper Listener → Voice Intents: executes transcript
voice/whisper_listener.py - Assistant Runtime → Voice Intents: starts listener with
main.py - Voice Intents → System Controller: invokes desktop actions
voice/speech_engine.py - Assistant Runtime → System Controller: dispatches gesture actions
main.py - System Controller → Windows Desktop: controls cursor and keys
controller/os_actions.py - System Controller → Windows Desktop: sets system volume
controller/os_actions.py - Assistant Runtime → HUD and Dwell: renders feedback
README.md - Assistant Runtime → Access Profiles: reads runtime settings
main.py - Voice Intents → Access Profiles: applies voice profiles
voice/speech_engine.py - User → Assistant Runtime: starts assistant
README.md