# mubai-he/chatwaifu-next architecture

> Architecture diagram of the GitHub repository mubai-he/chatwaifu-next, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mubai-he/chatwaifu-next
- Repository: https://github.com/mubai-he/chatwaifu-next

## Overview

ChatWaifu NEXT is a cross-platform character-companion product: users chat or speak through a Web Galgame or desktop pet, while the shared Runtime handles conversation, character behavior, memory, skills, and audio. Responses and avatar cues return to the client for playback and rendering. The README documents the major subsystem boundaries and high-level flow; sampled source confirms selected client, API, and avatar behavior, but not every service-to-service call, so uncertain wiring is omitted.

Main entry points: [`services/runtime/src/chatwaifu_runtime/conversation/service.py`](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/conversation/service.py) (Conversation service), [`services/runtime/src/chatwaifu_runtime/api/routes.py`](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/api/routes.py) (Runtime API), [`apps/web/src/features/chat/useChatSession.ts`](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/chat/useChatSession.ts) (Chat session), [`apps/web/src/features/desktop-pet/DesktopPetPage.tsx`](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/desktop-pet/DesktopPetPage.tsx) (Desktop pet)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["User clients"]
  node_web_app["Web Galgame<br/>[WebProductApp.tsx]"]
  node_desktop_app["Desktop pet<br/>[DesktopPetPage.tsx]"]
  node_chat_session["Chat session<br/>[useChatSession.ts]"]
  node_runtime_clients["Runtime clients<br/>[runtimeClient.ts]"]
  node_desktop_host["Tauri host<br/>[lib.rs]"]
end

subgraph group_runtime["Runtime core"]
  node_runtime_app["Runtime app<br/>[main.py]"]
  node_api_routes["Runtime API<br/>[routes.py]"]
  node_conversation["Conversation service<br/>[service.py]"]
end

subgraph group_capabilities["Companion capabilities"]
  node_character_kernel["Character kernel<br/>[service.py]"]
  node_memory["Structured memory<br/>[service.py]"]
  node_skills["Skills and MCP<br/>[service.py]"]
  node_realtime["Realtime audio<br/>[service.py]"]
  node_personal_assistant["Personal assistant<br/>[agenda.py]"]
  node_external_channels["External channels<br/>[service.py]"]
  node_photo_memory["Photo memory<br/>[recall.py]"]
  node_sticker_library["Sticker library<br/>[service.py]"]
end

subgraph group_state_integrations["State and integrations"]
  node_providers["Model and voice providers<br/>[factory.py]"]
  node_workers["Local model workers<br/>[main.py]"]
  node_database[("SQLite state<br/>[database.py]")]
  node_google_services["Google Calendar and Tasks"]
end

subgraph group_avatar["Avatar rendering"]
  node_avatar_controller["Avatar SDK<br/>[controller.ts]"]
  node_renderer["Live2D or Fake renderer<br/>[renderer.ts]"]
end

node_user(("User"))
node_external_models["Hosted model services"]

node_user -->|"uses"| node_web_app
node_user -->|"uses"| node_desktop_app
node_web_app -->|"opens chat"| node_chat_session
node_desktop_app -->|"opens chat"| node_chat_session
node_desktop_app -->|"uses host"| node_desktop_host
node_chat_session -->|"sends requests"| node_runtime_clients
node_runtime_clients -->|"HTTP and WS"| node_api_routes
node_runtime_app -->|"mounts routes"| node_api_routes
node_api_routes -->|"handles turns"| node_conversation
node_conversation -->|"uses character"| node_character_kernel
node_conversation -->|"uses memory"| node_memory
node_conversation -->|"dispatches skills"| node_skills
node_conversation -->|"requests inference"| node_providers
node_conversation -->|"coordinates audio"| node_realtime
node_api_routes -->|"routes assistant actions"| node_personal_assistant
node_api_routes -->|"routes channel actions"| node_external_channels
node_api_routes -->|"routes photo actions"| node_photo_memory
node_api_routes -->|"routes sticker actions"| node_sticker_library
node_providers -.->|"dispatches local work"| node_workers
node_providers -.->|"calls hosted models"| node_external_models
node_personal_assistant -.->|"integrates calendar"| node_google_services
node_conversation -->|"persists state"| node_database
node_memory -->|"reads and writes"| node_database
node_skills -->|"stores skill state"| node_database
node_chat_session -->|"applies avatar cues"| node_avatar_controller
node_avatar_controller -->|"updates frames"| node_renderer
node_avatar_controller -->|"emits interactions"| node_chat_session

click node_web_app "https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/product/web/WebProductApp.tsx"
click node_desktop_app "https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/desktop-pet/DesktopPetPage.tsx"
click node_chat_session "https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/chat/useChatSession.ts"
click node_runtime_clients "https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/chat/runtimeClient.ts"
click node_desktop_host "https://github.com/mubai-he/chatwaifu-next/blob/main/apps/desktop/src-tauri/src/lib.rs"
click node_runtime_app "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/main.py"
click node_api_routes "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/api/routes.py"
click node_conversation "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/conversation/service.py"
click node_character_kernel "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/character_kernel/service.py"
click node_memory "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/memory/service.py"
click node_skills "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/runtime_skills/service.py"
click node_realtime "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/realtime/service.py"
click node_personal_assistant "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/personal_assistant/agenda.py"
click node_external_channels "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/external_channels/service.py"
click node_photo_memory "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/photo_memory/recall.py"
click node_sticker_library "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/sticker_library/service.py"
click node_providers "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/providers/factory.py"
click node_workers "https://github.com/mubai-he/chatwaifu-next/blob/main/workers/tts-neural/src/chatwaifu_tts_neural_worker/main.py"
click node_database "https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/persistence/database.py"
click node_avatar_controller "https://github.com/mubai-he/chatwaifu-next/blob/main/packages/avatar-sdk/src/controller.ts"
click node_renderer "https://github.com/mubai-he/chatwaifu-next/blob/main/packages/avatar-sdk/src/renderer.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web_app,node_desktop_app,node_chat_session,node_runtime_clients,node_desktop_host,node_user toneBlue
class node_runtime_app,node_api_routes,node_conversation toneAmber
class node_character_kernel,node_memory,node_skills,node_realtime,node_personal_assistant,node_external_channels,node_photo_memory,node_sticker_library toneMint
class node_providers,node_workers,node_database,node_google_services toneRose
class node_avatar_controller,node_renderer,node_external_models toneIndigo
```

## Components

### User clients

- [Web Galgame](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/product/web/WebProductApp.tsx): `apps/web/src/product/web/WebProductApp.tsx`
- [Desktop pet](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/desktop-pet/DesktopPetPage.tsx): `apps/web/src/features/desktop-pet/DesktopPetPage.tsx`
- [Chat session](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/chat/useChatSession.ts): `apps/web/src/features/chat/useChatSession.ts`
- [Runtime clients](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/web/src/features/chat/runtimeClient.ts): `apps/web/src/features/chat/runtimeClient.ts`
- [Tauri host](https://github.com/mubai-he/chatwaifu-next/blob/main/apps/desktop/src-tauri/src/lib.rs): `apps/desktop/src-tauri/src/lib.rs`

### Runtime core

- [Runtime app](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/main.py): `services/runtime/src/chatwaifu_runtime/main.py`
- [Runtime API](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/api/routes.py): `services/runtime/src/chatwaifu_runtime/api/routes.py`
- [Conversation service](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/conversation/service.py): `services/runtime/src/chatwaifu_runtime/conversation/service.py`

### Companion capabilities

- [Character kernel](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/character_kernel/service.py): `services/runtime/src/chatwaifu_runtime/character_kernel/service.py`
- [Structured memory](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/memory/service.py): `services/runtime/src/chatwaifu_runtime/memory/service.py`
- [Skills and MCP](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/runtime_skills/service.py): `services/runtime/src/chatwaifu_runtime/runtime_skills/service.py`
- [Realtime audio](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/realtime/service.py): `services/runtime/src/chatwaifu_runtime/realtime/service.py`
- [Personal assistant](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/personal_assistant/agenda.py): `services/runtime/src/chatwaifu_runtime/personal_assistant/agenda.py`
- [External channels](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/external_channels/service.py): `services/runtime/src/chatwaifu_runtime/external_channels/service.py`
- [Photo memory](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/photo_memory/recall.py): `services/runtime/src/chatwaifu_runtime/photo_memory/recall.py`
- [Sticker library](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/sticker_library/service.py): `services/runtime/src/chatwaifu_runtime/sticker_library/service.py`

### State and integrations

- [Model and voice providers](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/providers/factory.py): `services/runtime/src/chatwaifu_runtime/providers/factory.py`
- [Local model workers](https://github.com/mubai-he/chatwaifu-next/blob/main/workers/tts-neural/src/chatwaifu_tts_neural_worker/main.py): `workers/tts-neural/src/chatwaifu_tts_neural_worker/main.py`
- [SQLite state](https://github.com/mubai-he/chatwaifu-next/blob/main/services/runtime/src/chatwaifu_runtime/persistence/database.py): `services/runtime/src/chatwaifu_runtime/persistence/database.py`
- **Google Calendar and Tasks**

### Avatar rendering

- [Avatar SDK](https://github.com/mubai-he/chatwaifu-next/blob/main/packages/avatar-sdk/src/controller.ts): `packages/avatar-sdk/src/controller.ts`
- [Live2D or Fake renderer](https://github.com/mubai-he/chatwaifu-next/blob/main/packages/avatar-sdk/src/renderer.ts): `packages/avatar-sdk/src/renderer.ts`

### Other components

- **User**
- **Hosted model services**

## Connections

- User → Web Galgame: uses
- User → Desktop pet: uses
- Web Galgame → Chat session: opens chat
- Desktop pet → Chat session: opens chat
- Desktop pet → Tauri host: uses host
- Chat session → Runtime clients: sends requests
- Runtime clients → Runtime API: HTTP and WS
- Runtime app → Runtime API: mounts routes
- Runtime API → Conversation service: handles turns
- Conversation service → Character kernel: uses character
- Conversation service → Structured memory: uses memory
- Conversation service → Skills and MCP: dispatches skills
- Conversation service → Model and voice providers: requests inference
- Conversation service → Realtime audio: coordinates audio
- Runtime API → Personal assistant: routes assistant actions
- Runtime API → External channels: routes channel actions
- Runtime API → Photo memory: routes photo actions
- Runtime API → Sticker library: routes sticker actions
- Model and voice providers → Local model workers: dispatches local work
- Model and voice providers → Hosted model services: calls hosted models
- Personal assistant → Google Calendar and Tasks: integrates calendar
- Conversation service → SQLite state: persists state
- Structured memory → SQLite state: reads and writes
- Skills and MCP → SQLite state: stores skill state
- Chat session → Avatar SDK: applies avatar cues
- Avatar SDK → Live2D or Fake renderer: updates frames
- Avatar SDK → Chat session: emits interactions