How mubai-he/chatwaifu-next works
ChatWaifu NEXT is a cross-platform character-companion product: users chat or speak through a Web Galgame or desktop pet, while the shared Runtime handles conversation, character behavior, memory, skills, and audio. Responses and avatar cues return to the client for playback and rendering. The README documents the major subsystem boundaries and high-level flow; sampled source confirms selected client, API, and avatar behavior, but not every service-to-service call, so uncertain wiring is omitted.
Main entry points: service.py Conversation service · routes.py Runtime API · useChatSession.ts Chat session · DesktopPetPage.tsx Desktop pet
Components 24
User clients
- Web Galgame
apps/web/src/product/web/WebProductApp.tsx - Desktop pet
apps/web/src/features/desktop-pet/DesktopPetPage.tsx - Chat session
apps/web/src/features/chat/useChatSession.ts - Runtime clients
apps/web/src/features/chat/runtimeClient.ts - Tauri host
apps/desktop/src-tauri/src/lib.rs
Runtime core
- Runtime app
services/runtime/src/chatwaifu_runtime/main.py - Runtime API
services/runtime/src/chatwaifu_runtime/api/routes.py - Conversation service
services/runtime/src/chatwaifu_runtime/conversation/service.py
Companion capabilities
- Character kernel
services/runtime/src/chatwaifu_runtime/character_kernel/service.py - Structured memory
services/runtime/src/chatwaifu_runtime/memory/service.py - Skills and MCP
services/runtime/src/chatwaifu_runtime/runtime_skills/service.py - Realtime audio
services/runtime/src/chatwaifu_runtime/realtime/service.py - Personal assistant
services/runtime/src/chatwaifu_runtime/personal_assistant/agenda.py - External channels
services/runtime/src/chatwaifu_runtime/external_channels/service.py - Photo memory
services/runtime/src/chatwaifu_runtime/photo_memory/recall.py - Sticker library
services/runtime/src/chatwaifu_runtime/sticker_library/service.py
State and integrations
- Model and voice providers
services/runtime/src/chatwaifu_runtime/providers/factory.py - Local model workers
workers/tts-neural/src/chatwaifu_tts_neural_worker/main.py - SQLite state
services/runtime/src/chatwaifu_runtime/persistence/database.py - Google Calendar and Tasks
Avatar rendering
- Avatar SDK
packages/avatar-sdk/src/controller.ts - Live2D or Fake renderer
packages/avatar-sdk/src/renderer.ts
Other components
- User
- Hosted model services
Connections 27
- User → Web Galgame: uses
- User → Desktop pet: uses
- Web Galgame → Chat session: opens chat
- Desktop pet → Chat session: opens chat
- Desktop pet → Tauri host: uses host
- Chat session → Runtime clients: sends requests
- Runtime clients → Runtime API: HTTP and WS
- Runtime app → Runtime API: mounts routes
- Runtime API → Conversation service: handles turns
- Conversation service → Character kernel: uses character
- Conversation service → Structured memory: uses memory
- Conversation service → Skills and MCP: dispatches skills
- Conversation service → Model and voice providers: requests inference
- Conversation service → Realtime audio: coordinates audio
- Runtime API → Personal assistant: routes assistant actions
- Runtime API → External channels: routes channel actions
- Runtime API → Photo memory: routes photo actions
- Runtime API → Sticker library: routes sticker actions
- Model and voice providers → Local model workers: dispatches local work
- Model and voice providers → Hosted model services: calls hosted models
- Personal assistant → Google Calendar and Tasks: integrates calendar
- Conversation service → SQLite state: persists state
- Structured memory → SQLite state: reads and writes
- Skills and MCP → SQLite state: stores skill state
- Chat session → Avatar SDK: applies avatar cues
- Avatar SDK → Live2D or Fake renderer: updates frames
- Avatar SDK → Chat session: emits interactions