# jofizcd/soul-of-waifu architecture

> Architecture diagram of the GitHub repository jofizcd/soul-of-waifu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jofizcd/soul-of-waifu
- Repository: https://github.com/jofizcd/soul-of-waifu

## Overview

#### Purpose and entry points
Soul of Waifu is a Windows-first Python desktop application for persistent AI characters across chat/voice, RPG, desktop companion, Discord, and LAN web access. Startup is `main.py`; Windows launch/install wrappers are `start.bat` and `installer.bat`. Runtime settings and character/provider metadata are persisted under `app/configuration/`.

#### Desktop application shell
The GUI is the primary orchestration boundary: it presents chat, settings, character workflows, calls, and feature modes, then routes user actions through signals into service modules. Core UI/system paths are `app/gui/sowInterface.py`, `app/gui/sowSystem.py`, and `app/gui/interface_signals.py`. `app/gui/soul_stage_page.py` provides the dedicated RPG view.

#### AI generation abstraction
`app/utils/ai_clients/ai_factory.py` selects an LLM provider behind the common contract in `app/utils/ai_clients/base_provider.py`. Provider implementations span local inference and cloud APIs in `app/utils/ai_clients/providers/`, while `app/utils/ai_clients/local_server_manager.py` manages the local Llama.cpp-style server lifecycle. Prompt construction is centralized in `app/utils/ai_clients/prompt_engine.py`, separating character/context assembly from model transport.

#### Persistent character cognition
Long-lived per-character continuity is handled by `app/utils/soul_memory.py`: psychology, relationship state, episodic retrieval, diary, emotional decay, and contradiction repair. Character-card import/export and character representation are managed in `app/utils/character_cards.py`. This memory layer feeds prompts for normal chat and can synchronize context into other modes rather than being owned by any one UI surface.

#### Structured RPG mode
Soul Stage is a separately modeled AI game-master flow implemented by `app/utils/ai_clients/soul_stage_engine.py` and displayed through `app/gui/soul_stage_page.py`. It converts player/party actions into structured campaign state—world facts, NPCs, inventory, status, choices, background and ambient changes—while using the AI client boundary for GM generation. It shares character identity/memory but retains RPG-specific state control.

#### Companion and external interaction boundary
The desktop-overlay companion is centered on `app/utils/soul_companion/soul_companion.py`; plugin extension points live in `app/utils/soul_companion/plugins/`. MCP integrations are isolated in `app/utils/ai_clients/mcp_client.py`, creating an explicit boundary between model-directed actions and external tool servers. The companion can use local desktop capabilities such as screen/clipboard/web behavior and proactively react through its behavioral simulation.

#### Multimodal and remote channels
Voice pipeline services are `app/utils/speech_to_text.py` and `app/utils/text_to_speech.py`; they bridge microphone/audio playback with local or cloud speech engines. Image-generation adapters reside in `app/utils/image_generator.py`. Discord is a distinct gateway via `app/utils/discord_manager.py`. LAN/mobile access is served by `app/utils/web_server.py`, with a browser client in `app/web_client/index.html` and `app/web_client/script.js`, providing WebSocket-style bidirectional interaction against the desktop-hosted runtime.

#### Runtime assets and infrastructure
The app expects writable/downloaded runtime areas for models, FFmpeg, voices, data, and cache: `assets/local_llm/`, `app/ffmpeg/`, `app/voices/`, `app/data/`, and `app/cache/`. Avatar/emotion assets are organized under `assets/emotions/`, including Live2D and VRM resources. Local-first execution supports GPU-accelerated model serving; optional external services include OpenAI-compatible and named cloud LLM providers, image APIs, ElevenLabs, Discord, Hugging Face model discovery/downloads (`app/utils/models_hub.py`), and external MCP servers.

Main entry points: [`app/gui/sowInterface.py`](https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/sowInterface.py) (Desktop UI orchestration), [`app/utils/ai_clients/prompt_engine.py`](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/prompt_engine.py) (Prompt engine), [`app/utils/ai_clients/soul_stage_engine.py`](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/soul_stage_engine.py) (Soul Stage engine), [`app/utils/soul_companion/soul_companion.py`](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/soul_companion/soul_companion.py) (Desktop companion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_desktop["Desktop shell"]
  node_startup["Application startup<br/>Python entry point<br/>[main.py]"]
  node_gui["Desktop UI orchestration<br/>Qt application shell<br/>[sowInterface.py]"]
  node_ui_system["UI system layer<br/>GUI system<br/>[sowSystem.py]"]
  node_ui_signals["UI signal routing<br/>signal boundary"]
end

subgraph group_ai["AI generation"]
  node_ai_factory{{"AI provider factory<br/>LLM selector<br/>[ai_factory.py]"}}
  node_provider_contract["Provider contract<br/>LLM interface<br/>[base_provider.py]"]
  node_providers["LLM providers<br/>local and cloud adapters"]
  node_prompt_engine["Prompt engine<br/>context assembler<br/>[prompt_engine.py]"]
  node_local_server["Local model server<br/>Llama.cpp lifecycle manager"]
end

subgraph group_experience["Modes &amp; channels"]
  node_soul_stage_ui["Soul Stage view<br/>RPG UI<br/>[soul_stage_page.py]"]
  node_soul_stage["Soul Stage engine<br/>RPG game master"]
  node_companion["Desktop companion<br/>overlay runtime<br/>[soul_companion.py]"]
  node_companion_plugins["Companion plugins<br/>extension points"]
  node_mcp{{"MCP client<br/>tool-server boundary<br/>[mcp_client.py]"}}
  node_voice["Voice pipeline<br/>speech I/O<br/>[speech_to_text.py]"]
  node_discord["Discord gateway<br/>remote channel<br/>[discord_manager.py]"]
  node_lan_web["LAN web gateway<br/>desktop-hosted web server<br/>[web_server.py]"]
  node_browser_client["Browser client<br/>LAN/mobile UI<br/>[index.html]"]
end

subgraph group_state["State &amp; assets"]
  node_memory[("Character memory<br/>persistent cognition<br/>[soul_memory.py]")]
  node_character_cards["Character cards<br/>character identity<br/>[character_cards.py]"]
  node_runtime_state[("Runtime configuration &amp; data<br/>writable state")]
end

subgraph group_external["External services"]
  node_cloud_services(("Cloud AI services<br/>optional providers"))
end

node_startup -->|"starts"| node_gui
node_gui -->|"uses"| node_ui_system
node_gui -->|"routes actions through"| node_ui_signals
node_ui_signals -->|"chat requests"| node_prompt_engine
node_prompt_engine -->|"reads context"| node_memory
node_prompt_engine -->|"uses identity"| node_character_cards
node_prompt_engine -->|"submits prompt"| node_ai_factory
node_ai_factory -->|"targets"| node_provider_contract
node_provider_contract -->|"implemented by"| node_providers
node_providers -->|"local inference"| node_local_server
node_providers -->|"cloud APIs"| node_cloud_services
node_memory -->|"persists"| node_runtime_state
node_character_cards -->|"stores metadata"| node_runtime_state
node_gui -->|"opens"| node_soul_stage_ui
node_soul_stage_ui -->|"player actions"| node_soul_stage
node_soul_stage -->|"GM generation"| node_prompt_engine
node_soul_stage -.->|"shares continuity"| node_memory
node_gui -->|"controls"| node_companion
node_companion -->|"extends with"| node_companion_plugins
node_companion -->|"external actions"| node_mcp
node_mcp -.->|"tool servers"| node_cloud_services
node_gui -->|"calls and chat audio"| node_voice
node_voice -.->|"optional speech engines"| node_cloud_services
node_gui -->|"connects"| node_discord
node_discord -->|"messages"| node_prompt_engine
node_lan_web -->|"serves"| node_browser_client
node_browser_client -->|"bidirectional interaction"| node_lan_web
node_lan_web -->|"remote chat"| node_prompt_engine

click node_startup "https://github.com/jofizcd/soul-of-waifu/blob/main/main.py"
click node_gui "https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/sowInterface.py"
click node_ui_system "https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/sowSystem.py"
click node_ui_signals "https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/interface_signals.py"
click node_ai_factory "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/ai_factory.py"
click node_provider_contract "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/base_provider.py"
click node_prompt_engine "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/prompt_engine.py"
click node_local_server "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/local_server_manager.py"
click node_memory "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/soul_memory.py"
click node_character_cards "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/character_cards.py"
click node_soul_stage_ui "https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/soul_stage_page.py"
click node_soul_stage "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/soul_stage_engine.py"
click node_companion "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/soul_companion/soul_companion.py"
click node_mcp "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/mcp_client.py"
click node_voice "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/speech_to_text.py"
click node_discord "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/discord_manager.py"
click node_lan_web "https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/web_server.py"
click node_browser_client "https://github.com/jofizcd/soul-of-waifu/blob/main/app/web_client/index.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_startup,node_gui,node_ui_system,node_ui_signals toneBlue
class node_ai_factory,node_provider_contract,node_providers,node_prompt_engine,node_local_server toneAmber
class node_soul_stage_ui,node_soul_stage,node_companion,node_companion_plugins,node_mcp,node_voice,node_discord,node_lan_web,node_browser_client toneMint
class node_memory,node_character_cards,node_runtime_state toneRose
class node_cloud_services toneIndigo
```

## Components

### Desktop shell

- [Application startup](https://github.com/jofizcd/soul-of-waifu/blob/main/main.py): Python entry point · `main.py`
- [Desktop UI orchestration](https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/sowInterface.py): Qt application shell · Routes UI actions across chat, settings, calls, and modes. · `app/gui/sowInterface.py`
- [UI system layer](https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/sowSystem.py): GUI system · `app/gui/sowSystem.py`
- [UI signal routing](https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/interface_signals.py): signal boundary · `app/gui/interface_signals.py`

### AI generation

- [AI provider factory](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/ai_factory.py): LLM selector · `app/utils/ai_clients/ai_factory.py`
- [Provider contract](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/base_provider.py): LLM interface · `app/utils/ai_clients/base_provider.py`
- **LLM providers**: local and cloud adapters
- [Prompt engine](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/prompt_engine.py): context assembler · Builds character and conversation context before model transport. · `app/utils/ai_clients/prompt_engine.py`
- [Local model server](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/local_server_manager.py): Llama.cpp lifecycle manager · `app/utils/ai_clients/local_server_manager.py`

### Modes & channels

- [Soul Stage view](https://github.com/jofizcd/soul-of-waifu/blob/main/app/gui/soul_stage_page.py): RPG UI · `app/gui/soul_stage_page.py`
- [Soul Stage engine](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/soul_stage_engine.py): RPG game master · Controls campaign facts, NPCs, inventory, status, choices, and ambience. · `app/utils/ai_clients/soul_stage_engine.py`
- [Desktop companion](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/soul_companion/soul_companion.py): overlay runtime · Runs proactive desktop behavior and local capabilities. · `app/utils/soul_companion/soul_companion.py`
- **Companion plugins**: extension points
- [MCP client](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/ai_clients/mcp_client.py): tool-server boundary · Isolates model-directed actions from external tool servers. · `app/utils/ai_clients/mcp_client.py`
- [Voice pipeline](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/speech_to_text.py): speech I/O · Bridges microphone and playback with local or cloud speech engines. · `app/utils/speech_to_text.py`
- [Discord gateway](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/discord_manager.py): remote channel · `app/utils/discord_manager.py`
- [LAN web gateway](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/web_server.py): desktop-hosted web server · Provides bidirectional browser interaction with the desktop runtime. · `app/utils/web_server.py`
- [Browser client](https://github.com/jofizcd/soul-of-waifu/blob/main/app/web_client/index.html): LAN/mobile UI · `app/web_client/index.html`

### State & assets

- [Character memory](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/soul_memory.py): persistent cognition · Maintains psychology, relationships, episodic memory, diary, and emotional state. · `app/utils/soul_memory.py`
- [Character cards](https://github.com/jofizcd/soul-of-waifu/blob/main/app/utils/character_cards.py): character identity · `app/utils/character_cards.py`
- **Runtime configuration & data**: writable state · Stores settings, metadata, data, cache, models, voices, and media tooling.

### External services

- **Cloud AI services**: optional providers · LLM, speech, image, Discord, model discovery, and MCP services.

## Connections

- Application startup → Desktop UI orchestration: starts
- Desktop UI orchestration → UI system layer: uses
- Desktop UI orchestration → UI signal routing: routes actions through
- UI signal routing → Prompt engine: chat requests
- Prompt engine → Character memory: reads context
- Prompt engine → Character cards: uses identity
- Prompt engine → AI provider factory: submits prompt
- AI provider factory → Provider contract: targets
- Provider contract → LLM providers: implemented by
- LLM providers → Local model server: local inference
- LLM providers → Cloud AI services: cloud APIs
- Character memory → Runtime configuration & data: persists
- Character cards → Runtime configuration & data: stores metadata
- Desktop UI orchestration → Soul Stage view: opens
- Soul Stage view → Soul Stage engine: player actions
- Soul Stage engine → Prompt engine: GM generation
- Soul Stage engine → Character memory: shares continuity
- Desktop UI orchestration → Desktop companion: controls
- Desktop companion → Companion plugins: extends with
- Desktop companion → MCP client: external actions
- MCP client → Cloud AI services: tool servers
- Desktop UI orchestration → Voice pipeline: calls and chat audio
- Voice pipeline → Cloud AI services: optional speech engines
- Desktop UI orchestration → Discord gateway: connects
- Discord gateway → Prompt engine: messages
- LAN web gateway → Browser client: serves
- Browser client → LAN web gateway: bidirectional interaction
- LAN web gateway → Prompt engine: remote chat