# 3-1-choiinsawman/gesture_ai architecture

> Architecture diagram of the GitHub repository 3-1-choiinsawman/gesture_ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/3-1-choiinsawman/gesture_ai
- Repository: https://github.com/3-1-choiinsawman/gesture_ai

## Overview

gesture_AI is a FastAPI server for real-time sign-language translation and streaming Korean speech transcription. The primary workflow receives 88-dimensional keypoint frames over WebSocket, validates and classifies them, stabilizes word predictions, detects utterance boundaries, and turns candidate words into a Korean sentence. A separate WebSocket handles audio transcription. The README documents these capabilities; sampled code confirms the main routing, classifier, and STT behavior. Summary-related code is commented out, so it is not represented as an active product flow.

Main entry points: [`router/cc.py`](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py) (Sign WebSocket), [`router/cc_stt.py`](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc_stt.py) (Audio WebSocket), [`main.py`](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/main.py) (FastAPI Application), [`util/service/cc_service.py`](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/service/cc_service.py) (Sentence Generation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and Sessions"]
  node_app["FastAPI Application<br/>[main.py]"]
  node_health["Health Routes<br/>[root.py]"]
end

subgraph group_sign["Sign Recognition"]
  node_sign_ws["Sign WebSocket<br/>[cc.py]"]
  node_frame_checks["Frame Validation<br/>[cc.py]"]
  node_prediction["Prediction Stabilization<br/>[cc.py]"]
  node_bigru["BiGRU Classifier<br/>[bigruClassifier.py]"]
end

subgraph group_language["Language Generation"]
  node_sentence_api["Sentence API Schema<br/>[subtitleSchema.py]"]
  node_sentence_service["Sentence Generation<br/>[cc_service.py]"]
end

subgraph group_speech["Speech Transcription"]
  node_stt_ws["Audio WebSocket<br/>[cc_stt.py]"]
  node_stt_state["Streaming Audio State<br/>[cc_stt.py]"]
end

subgraph group_shared["Shared Utilities"]
  node_config["Runtime Configuration<br/>[config.py]"]
  node_ssl["TLS Trust Setup<br/>[ssl_config.py]"]
  node_logger["Application Logger<br/>[loadLogger.py]"]
  node_message_loader["Message Loader<br/>[loadMessage.py]"]
  node_mongo_connect["Mongo Connection<br/>[mongo_connect.py]"]
end

node_client(("Streaming Client"))
node_ollama["Local Ollama"]
node_whisper["Faster-Whisper"]
node_mongo[("MongoDB")]

node_client -->|"streams keypoints"| node_sign_ws
node_app -->|"registers route"| node_sign_ws
node_app -->|"registers route"| node_stt_ws
node_app -->|"registers route"| node_health
node_app -->|"warms model"| node_sentence_service
node_sign_ws -->|"validates frames"| node_frame_checks
node_sign_ws -->|"processes frames"| node_prediction
node_prediction -->|"classifies sequences"| node_bigru
node_sign_ws -->|"reads settings"| node_config
node_sign_ws -->|"uses response schema"| node_sentence_api
node_sign_ws -->|"requests sentence"| node_sentence_service
node_sentence_service -->|"sends chat request"| node_ollama
node_sign_ws -->|"sends predictions"| node_client
node_client -->|"streams audio"| node_stt_ws
node_stt_ws -->|"updates session state"| node_stt_state
node_stt_ws -->|"reads settings"| node_config
node_stt_ws -->|"configures trust"| node_ssl
node_stt_state -->|"transcribes audio"| node_whisper
node_stt_ws -->|"sends text"| node_client
node_message_loader -->|"uses collection"| node_mongo_connect
node_mongo_connect -.->|"connects to"| node_mongo
node_sentence_service -->|"logs activity"| node_logger

click node_app "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/main.py"
click node_health "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/root.py"
click node_sign_ws "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py"
click node_frame_checks "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py"
click node_prediction "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py"
click node_bigru "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/bigruClassifier.py"
click node_config "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/config.py"
click node_sentence_api "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/schema/subtitleSchema.py"
click node_sentence_service "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/service/cc_service.py"
click node_stt_ws "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc_stt.py"
click node_stt_state "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc_stt.py"
click node_ssl "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/ssl_config.py"
click node_logger "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/loadLogger.py"
click node_message_loader "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/loadMessage.py"
click node_mongo_connect "https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/mongo_connect.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_health,node_client toneBlue
class node_sign_ws,node_frame_checks,node_prediction,node_bigru,node_mongo toneAmber
class node_sentence_api,node_sentence_service toneMint
class node_stt_ws,node_stt_state toneRose
class node_config,node_ssl,node_logger,node_message_loader,node_mongo_connect,node_ollama,node_whisper toneIndigo
```

## Components

### API and Sessions

- [FastAPI Application](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/main.py): `main.py`
- [Health Routes](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/root.py): `router/root.py`

### Sign Recognition

- [Sign WebSocket](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py): `router/cc.py`
- [Frame Validation](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py): `router/cc.py`
- [Prediction Stabilization](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc.py): `router/cc.py`
- [BiGRU Classifier](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/bigruClassifier.py): `util/bigruClassifier.py`

### Language Generation

- [Sentence API Schema](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/schema/subtitleSchema.py): `schema/subtitleSchema.py`
- [Sentence Generation](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/service/cc_service.py): `util/service/cc_service.py`

### Speech Transcription

- [Audio WebSocket](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc_stt.py): `router/cc_stt.py`
- [Streaming Audio State](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/router/cc_stt.py): `router/cc_stt.py`

### Shared Utilities

- [Runtime Configuration](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/config.py): `util/config.py`
- [TLS Trust Setup](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/ssl_config.py): `util/ssl_config.py`
- [Application Logger](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/loadLogger.py): `util/loadLogger.py`
- [Message Loader](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/loadMessage.py): `util/loadMessage.py`
- [Mongo Connection](https://github.com/3-1-choiinsawman/gesture_ai/blob/main/util/mongo_connect.py): `util/mongo_connect.py`

### Other components

- **Streaming Client**
- **Local Ollama**
- **Faster-Whisper**
- **MongoDB**

## Connections

- Streaming Client → Sign WebSocket: streams keypoints (evidence: `README.md`)
- FastAPI Application → Sign WebSocket: registers route (evidence: `main.py`)
- FastAPI Application → Audio WebSocket: registers route (evidence: `main.py`)
- FastAPI Application → Health Routes: registers route (evidence: `main.py`)
- FastAPI Application → Sentence Generation: warms model (evidence: `main.py`)
- Sign WebSocket → Frame Validation: validates frames (evidence: `router/cc.py`)
- Sign WebSocket → Prediction Stabilization: processes frames (evidence: `router/cc.py`)
- Prediction Stabilization → BiGRU Classifier: classifies sequences (evidence: `router/cc.py`)
- Sign WebSocket → Runtime Configuration: reads settings (evidence: `router/cc.py`)
- Sign WebSocket → Sentence API Schema: uses response schema (evidence: `router/cc.py`)
- Sign WebSocket → Sentence Generation: requests sentence (evidence: `router/cc.py`)
- Sentence Generation → Local Ollama: sends chat request (evidence: `util/service/cc_service.py`)
- Sign WebSocket → Streaming Client: sends predictions (evidence: `README.md`)
- Streaming Client → Audio WebSocket: streams audio (evidence: `README.md`)
- Audio WebSocket → Streaming Audio State: updates session state (evidence: `router/cc_stt.py`)
- Audio WebSocket → Runtime Configuration: reads settings (evidence: `router/cc_stt.py`)
- Audio WebSocket → TLS Trust Setup: configures trust (evidence: `router/cc_stt.py`)
- Streaming Audio State → Faster-Whisper: transcribes audio (evidence: `router/cc_stt.py`)
- Audio WebSocket → Streaming Client: sends text (evidence: `router/cc_stt.py`)
- Message Loader → Mongo Connection: uses collection (evidence: `util/loadMessage.py`)
- Mongo Connection → MongoDB: connects to
- Sentence Generation → Application Logger: logs activity (evidence: `util/service/cc_service.py`)
