# harsha-kyle/kyoma architecture

> Architecture diagram of the GitHub repository harsha-kyle/kyoma, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harsha-kyle/kyoma
- Repository: https://github.com/harsha-kyle/kyoma

## Overview

Local AI Vtuber 2 is a local-first conversational avatar: users chat through a web UI or provide screen/audio input; the backend handles input, LLM responses, memory and speech, while the frontend presents the conversation and character. Documented capabilities also include interruptible speech, vision, model and character management, editable/exportable history, and streaming. Edges cite the README or indexed source references; unsampled frontend-to-API wiring is left unasserted.

Main entry points: [`backend/server.py`](https://github.com/harsha-kyle/kyoma/blob/main/backend/server.py) (Backend API), [`frontend/src/lib/chatManager.ts`](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/chatManager.ts) (Chat manager), [`backend/services/TTS/TTS.py`](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/TTS/TTS.py) (Speech service), [`frontend/src/App.tsx`](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/App.tsx) (Web UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["User interaction"]
  node_webui["Web UI<br/>[App.tsx]"]
  node_chat["Chat manager<br/>[chatManager.ts]"]
  node_pipeline["Task pipeline<br/>[pipelineManager.ts]"]
  node_sessions[("Chat sessions<br/>[sessionManager.ts]")]
  node_export["History export<br/>[data-export.tsx]"]
end

subgraph group_backend["Backend orchestration"]
  node_api["Backend API<br/>[server.py]"]
  node_input["Voice input<br/>[Input.py]"]
  node_vision["Screen vision<br/>[VisionInput.py]"]
  node_chatfetch["Chat fetch<br/>[Chatfetch.py]"]
  node_automation["Desktop automation"]
end

subgraph group_intelligence["AI and memory"]
  node_llm["LLM routing<br/>[LLM.py]"]
  node_textllm["Text model<br/>[TextLLM.py]"]
  node_memory[("Long-term memory<br/>[Memory.py]")]
  node_history[("History store<br/>[HistoryStore.py]")]
end

subgraph group_avatar["Voice and avatar"]
  node_tts["Speech service<br/>[TTS.py]"]
  node_gptsovits["GPT-SoVITS<br/>[GptSovits.py]"]
  node_rvc["Voice conversion<br/>[rvc_server.py]"]
  node_character["Character models"]
  node_render["Avatar rendering"]
end

node_user(("User"))

node_user -->|"interacts with"| node_webui
node_webui -.->|"starts chat"| node_chat
node_chat -->|"tracks tasks"| node_pipeline
node_chat -->|"loads and saves"| node_sessions
node_user -.->|"provides input"| node_api
node_api -->|"routes audio"| node_input
node_api -->|"serves vision"| node_vision
node_api -->|"uses chat fetch"| node_chatfetch
node_api -->|"routes requests"| node_llm
node_api -->|"uses memory"| node_memory
node_api -->|"uses history"| node_history
node_api -->|"routes speech"| node_tts
node_api -->|"manages models"| node_character
node_llm -->|"selects model"| node_textllm
node_chat -.->|"supports export"| node_export
node_tts -.->|"uses provider"| node_gptsovits
node_tts -.->|"can convert voice"| node_rvc
node_user -.->|"controls desktop"| node_automation
node_character -.->|"supplies models"| node_render
node_user -.->|"can interrupt"| node_pipeline

click node_webui "https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/App.tsx"
click node_chat "https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/chatManager.ts"
click node_pipeline "https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/pipelineManager.ts"
click node_sessions "https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/sessionManager.ts"
click node_api "https://github.com/harsha-kyle/kyoma/blob/main/backend/server.py"
click node_input "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Input/Input.py"
click node_vision "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Input/VisionInput.py"
click node_chatfetch "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/ChatFetch/Chatfetch.py"
click node_automation "https://github.com/harsha-kyle/kyoma/blob/main/Kyoma/Kyoma_automation.py"
click node_llm "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/LLM/LLM.py"
click node_textllm "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/LLM/TextLLM.py"
click node_memory "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Memory/Memory.py"
click node_history "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Memory/HistoryStore.py"
click node_tts "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/TTS/TTS.py"
click node_gptsovits "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/TTS/GPTsovits/GptSovits.py"
click node_rvc "https://github.com/harsha-kyle/kyoma/blob/main/backend/plugins/rvc/rvc_server.py"
click node_character "https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Character/characterManager.py"
click node_render "https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/components/live-2d-renderer.tsx"
click node_export "https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/components/data-export.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webui,node_chat,node_pipeline,node_sessions,node_export,node_user toneBlue
class node_api,node_input,node_vision,node_chatfetch,node_automation toneAmber
class node_llm,node_textllm,node_memory,node_history toneMint
class node_tts,node_gptsovits,node_rvc,node_character,node_render toneRose
```

## Components

### User interaction

- [Web UI](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Chat manager](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/chatManager.ts): `frontend/src/lib/chatManager.ts`
- [Task pipeline](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/pipelineManager.ts): `frontend/src/lib/pipelineManager.ts`
- [Chat sessions](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/lib/sessionManager.ts): `frontend/src/lib/sessionManager.ts`
- [History export](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/components/data-export.tsx): `frontend/src/components/data-export.tsx`

### Backend orchestration

- [Backend API](https://github.com/harsha-kyle/kyoma/blob/main/backend/server.py): `backend/server.py`
- [Voice input](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Input/Input.py): `backend/services/Input/Input.py`
- [Screen vision](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Input/VisionInput.py): `backend/services/Input/VisionInput.py`
- [Chat fetch](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/ChatFetch/Chatfetch.py): `backend/services/ChatFetch/Chatfetch.py`
- [Desktop automation](https://github.com/harsha-kyle/kyoma/blob/main/Kyoma/Kyoma_automation.py): `Kyoma/Kyoma_automation.py`

### AI and memory

- [LLM routing](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/LLM/LLM.py): `backend/services/LLM/LLM.py`
- [Text model](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/LLM/TextLLM.py): `backend/services/LLM/TextLLM.py`
- [Long-term memory](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Memory/Memory.py): `backend/services/Memory/Memory.py`
- [History store](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Memory/HistoryStore.py): `backend/services/Memory/HistoryStore.py`

### Voice and avatar

- [Speech service](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/TTS/TTS.py): `backend/services/TTS/TTS.py`
- [GPT-SoVITS](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/TTS/GPTsovits/GptSovits.py): `backend/services/TTS/GPTsovits/GptSovits.py`
- [Voice conversion](https://github.com/harsha-kyle/kyoma/blob/main/backend/plugins/rvc/rvc_server.py): `backend/plugins/rvc/rvc_server.py`
- [Character models](https://github.com/harsha-kyle/kyoma/blob/main/backend/services/Character/characterManager.py): `backend/services/Character/characterManager.py`
- [Avatar rendering](https://github.com/harsha-kyle/kyoma/blob/main/frontend/src/components/live-2d-renderer.tsx): `frontend/src/components/live-2d-renderer.tsx`

### Other components

- **User**

## Connections

- User → Web UI: interacts with
- Web UI → Chat manager: starts chat
- Chat manager → Task pipeline: tracks tasks (evidence: `frontend/src/lib/chatManager.ts`)
- Chat manager → Chat sessions: loads and saves (evidence: `frontend/src/lib/chatManager.ts`)
- User → Backend API: provides input
- Backend API → Voice input: routes audio (evidence: `backend/server.py`)
- Backend API → Screen vision: serves vision (evidence: `backend/server.py`)
- Backend API → Chat fetch: uses chat fetch (evidence: `backend/server.py`)
- Backend API → LLM routing: routes requests (evidence: `backend/server.py`)
- Backend API → Long-term memory: uses memory (evidence: `backend/server.py`)
- Backend API → History store: uses history (evidence: `backend/server.py`)
- Backend API → Speech service: routes speech (evidence: `backend/server.py`)
- Backend API → Character models: manages models (evidence: `backend/server.py`)
- LLM routing → Text model: selects model (evidence: `backend/services/LLM/LLM.py`)
- Chat manager → History export: supports export
- Speech service → GPT-SoVITS: uses provider
- Speech service → Voice conversion: can convert voice
- User → Desktop automation: controls desktop
- Character models → Avatar rendering: supplies models
- User → Task pipeline: can interrupt
