How 3-1-choiinsawman/gesture_ai works
gesture_AI is a FastAPI server for real-time sign-language translation and streaming Korean speech transcription. The primary workflow receives 88-dimensional keypoint frames over WebSocket, validates and classifies them, stabilizes word predictions, detects utterance boundaries, and turns candidate words into a Korean sentence. A separate WebSocket handles audio transcription. The README documents these capabilities; sampled code confirms the main routing, classifier, and STT behavior. Summary-related code is commented out, so it is not represented as an active product flow.
Main entry points: cc.py Sign WebSocket · cc_stt.py Audio WebSocket · main.py FastAPI Application · cc_service.py Sentence Generation
Components 19
API and Sessions
- FastAPI Application
main.py - Health Routes
router/root.py
Sign Recognition
- Sign WebSocket
router/cc.py - Frame Validation
router/cc.py - Prediction Stabilization
router/cc.py - BiGRU Classifier
util/bigruClassifier.py
Language Generation
- Sentence API Schema
schema/subtitleSchema.py - Sentence Generation
util/service/cc_service.py
Speech Transcription
- Audio WebSocket
router/cc_stt.py - Streaming Audio State
router/cc_stt.py
Shared Utilities
- Runtime Configuration
util/config.py - TLS Trust Setup
util/ssl_config.py - Application Logger
util/loadLogger.py - Message Loader
util/loadMessage.py - Mongo Connection
util/mongo_connect.py
Other components
- Streaming Client
- Local Ollama
- Faster-Whisper
- MongoDB
Connections 22
- Streaming Client → Sign WebSocket: streams keypoints
README.md - FastAPI Application → Sign WebSocket: registers route
main.py - FastAPI Application → Audio WebSocket: registers route
main.py - FastAPI Application → Health Routes: registers route
main.py - FastAPI Application → Sentence Generation: warms model
main.py - Sign WebSocket → Frame Validation: validates frames
router/cc.py - Sign WebSocket → Prediction Stabilization: processes frames
router/cc.py - Prediction Stabilization → BiGRU Classifier: classifies sequences
router/cc.py - Sign WebSocket → Runtime Configuration: reads settings
router/cc.py - Sign WebSocket → Sentence API Schema: uses response schema
router/cc.py - Sign WebSocket → Sentence Generation: requests sentence
router/cc.py - Sentence Generation → Local Ollama: sends chat request
util/service/cc_service.py - Sign WebSocket → Streaming Client: sends predictions
README.md - Streaming Client → Audio WebSocket: streams audio
README.md - Audio WebSocket → Streaming Audio State: updates session state
router/cc_stt.py - Audio WebSocket → Runtime Configuration: reads settings
router/cc_stt.py - Audio WebSocket → TLS Trust Setup: configures trust
router/cc_stt.py - Streaming Audio State → Faster-Whisper: transcribes audio
router/cc_stt.py - Audio WebSocket → Streaming Client: sends text
router/cc_stt.py - Message Loader → Mongo Connection: uses collection
util/loadMessage.py - Mongo Connection → MongoDB: connects to
- Sentence Generation → Application Logger: logs activity
util/service/cc_service.py