How lyihub/ar-mahjong-assistant-preview works
ARmahjongAssist is an Android/RayNeo client backed by a FastAPI server for local Mahjong recognition, state tracking, and play advice. The main workflow captures a hand image, submits it with a session, detects tiles, updates game state, computes a recommendation, and returns it for display. A separate voice path transcribes audio and interprets game events to update visible-tile state. The README documents these stages; some Android internals and unsampled server implementation details are represented only at documented subsystem level.
Main entry points: main.py FastAPI Routes · MainActivity.kt RayNeo App · ChatViewModel.kt Chat ViewModel · ChatRepository.kt Chat Repository
Components 21
AR Client
- RayNeo App
app/src/main/java/com/example/ai_assist/MainActivity.kt - Chat ViewModel
app/src/main/java/com/example/ai_assist/viewmodel/ChatViewModel.kt - Chat Repository
app/src/main/java/com/example/ai_assist/repository/ChatRepository.kt - Game API Client
app/src/main/java/com/example/ai_assist/service/GameApiService.kt - Device Controls
app/src/main/java/com/example/ai_assist/service/RayNeoDeviceManager.kt - Audio Recorder
app/src/main/java/com/example/ai_assist/utils/RayNeoAudioRecorder.kt - Tile Mapper
app/src/main/java/com/example/ai_assist/utils/MahjongMapper.kt - Session Models
app/src/main/java/com/example/ai_assist/model/SessionModels.kt
Server API
- FastAPI Routes
server/main.py - Runtime Config
server/config.py - API Schemas
server/schemas.py - Session History
server/database.py
Tile Vision
- Vision Service
server/vision_service.py - ONNX Detection
server/yolo_inference.py
Game Analysis
- State Tracker
server/mahjong_state_tracker.py - Play Efficiency
server/efficiency_engine.py
Voice Events
- Speech Recognition
server/stt_service.py - Voice Intent
server/llm_service.py
Other components
- Player
- Local ONNX Model
server/models/yolo/weights.onnx - OpenAI-Compatible LLM
Connections 21
- Player → RayNeo App: operates
README.md - RayNeo App → Chat ViewModel: manages UI
- Chat ViewModel → Chat Repository: uses
- Chat Repository → Game API Client: requests
- RayNeo App → Device Controls: reads controls
- RayNeo App → Audio Recorder: records audio
- RayNeo App → FastAPI Routes: uploads hand
README.md - FastAPI Routes → Vision Service: detects tiles
server/main.py - Vision Service → ONNX Detection: runs inference
server/vision_service.py - ONNX Detection → Local ONNX Model: loads model
- FastAPI Routes → State Tracker: updates state
server/main.py - FastAPI Routes → Play Efficiency: calculates advice
README.md - FastAPI Routes → Session History: records sessions
server/main.py - FastAPI Routes → API Schemas: formats response
server/main.py - FastAPI Routes → RayNeo App: returns advice
README.md - Audio Recorder → FastAPI Routes: uploads audio
README.md - FastAPI Routes → Speech Recognition: transcribes audio
README.md - Speech Recognition → Voice Intent: passes transcript
README.md - Voice Intent → OpenAI-Compatible LLM: interprets events
- FastAPI Routes → State Tracker: applies events
README.md - FastAPI Routes → Runtime Config: uses settings
server/main.py