# nbllt666/cx-o architecture

> Architecture diagram of the GitHub repository nbllt666/cx-o, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nbllt666/cx-o
- Repository: https://github.com/nbllt666/cx-o

## Overview

CX-O is a desktop AI-companion system: users interact through a browser/Electron pet, which connects to a FastAPI backend for conversational and memory services; the frontend presents responses through voice and Live2D/VRM avatars. Major branches add streaming, meetings, screen perception, autonomy, and distributed failover. Separate workstations provide music/voice generation and model training, while CXO-Tuner supports optional self-evolution. Relationships for unsampled subsystems are included only where the README documents them; paths identify their enclosing source directories.

Main entry points: [`APP-Frontend/src`](https://github.com/nbllt666/cx-o/tree/main/APP-Frontend/src) (Pet and management UI), [`CX-O-SERVER/server/api`](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/api) (HTTP and WebSocket API), [`CX-O-SERVER/server/core/chat`](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/chat) (Conversation and tools), [`CX-O-VoiceWorkStation/workstation/services`](https://github.com/nbllt666/cx-o/tree/main/CX-O-VoiceWorkStation/workstation/services) (Voice workstation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Desktop experience"]
  node_frontend["Pet and management UI"]
  node_electron["Electron host<br/>[main.ts]"]
  node_avatar["Live2D and VRM"]
  node_capture["Screen perception"]
  node_meeting["Meeting experience"]
end

subgraph group_server["Companion backend"]
  node_backend["HTTP and WebSocket API"]
  node_chat["Conversation and tools"]
  node_memory[("Memory and retrieval")]
  node_asr["Speech recognition"]
  node_vision["Vision processing"]
  node_autonomy["Autonomous life<br/>[autonomy_engine.py]"]
  node_dream["Dream engine<br/>[engine.py]"]
end

subgraph group_media["Voice and music"]
  node_voice["Voice workstation"]
  node_song["Song generation<br/>[song_pipeline.py]"]
  node_duet["Duet conversion<br/>[duet_pipeline.py]"]
  node_modelstation["Model training<br/>[main.py]"]
end

subgraph group_operations["Management and evolution"]
  node_tuner["Evolution training<br/>[routes.py]"]
  node_admin["CX-A management"]
end

subgraph group_distributed["Integrations and resilience"]
  node_cluster["Sentinel cluster"]
end

node_user(("User"))
node_obs(("OBS"))
node_llm(("Ollama and LLMs"))
node_speech_services(("ASR and TTS services"))

node_user -->|"interacts"| node_frontend
node_frontend -->|"sends chat"| node_backend
node_backend -.->|"routes requests"| node_chat
node_chat -->|"retrieves memories"| node_memory
node_chat -->|"requests generation"| node_llm
node_backend -->|"uses speech recognition"| node_asr
node_frontend -->|"uses speech services"| node_speech_services
node_frontend -->|"renders character"| node_avatar
node_frontend -->|"uploads captured clips"| node_capture
node_capture -.->|"uploads perception data"| node_backend
node_backend -->|"processes screen input"| node_vision
node_frontend -->|"starts meetings"| node_meeting
node_meeting -->|"connects to meeting service"| node_backend
node_backend -->|"runs autonomous cycles"| node_autonomy
node_autonomy -.->|"starts dream sessions"| node_dream
node_frontend -->|"requests voice work"| node_voice
node_voice -.->|"generates songs"| node_song
node_voice -.->|"converts duet vocals"| node_duet
node_frontend -->|"manages voice models"| node_modelstation
node_frontend -->|"submits evolution feedback"| node_tuner
node_admin -->|"orchestrates nodes"| node_cluster
node_backend -->|"replicates state"| node_cluster
node_frontend -->|"provides streaming sources"| node_obs

click node_frontend "https://github.com/nbllt666/cx-o/tree/main/APP-Frontend/src"
click node_electron "https://github.com/nbllt666/cx-o/blob/main/APP-Frontend/electron/main.ts"
click node_avatar "https://github.com/nbllt666/cx-o/tree/main/APP-Frontend/src/avatar"
click node_capture "https://github.com/nbllt666/cx-o/blob/main/APP-Frontend/src/hooks/useVisionPipeline.ts"
click node_meeting "https://github.com/nbllt666/cx-o/blob/main/APP-Frontend/src/pages/pet/MeetingRoomView.tsx"
click node_backend "https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/api"
click node_chat "https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/chat"
click node_memory "https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/memory"
click node_asr "https://github.com/nbllt666/cx-o/blob/main/CX-O-SERVER/asr_container/streaming_engine.py"
click node_vision "https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/vision"
click node_autonomy "https://github.com/nbllt666/cx-o/blob/main/CX-O-SERVER/server/autonomy/core/loop/autonomy_engine.py"
click node_dream "https://github.com/nbllt666/cx-o/blob/main/CX-O-SERVER/server/autonomy/dream/engine.py"
click node_voice "https://github.com/nbllt666/cx-o/tree/main/CX-O-VoiceWorkStation/workstation/services"
click node_song "https://github.com/nbllt666/cx-o/blob/main/CX-O-VoiceWorkStation/workstation/services/song_pipeline.py"
click node_duet "https://github.com/nbllt666/cx-o/blob/main/CX-O-VoiceWorkStation/workstation/services/duet_pipeline.py"
click node_modelstation "https://github.com/nbllt666/cx-o/blob/main/CXO-ModelStation/modelstation/main.py"
click node_tuner "https://github.com/nbllt666/cx-o/blob/main/CXO-Tuner/tuner/api/routes.py"
click node_admin "https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/admin"
click node_cluster "https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/cluster"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend,node_electron,node_avatar,node_capture,node_meeting,node_user toneBlue
class node_backend,node_chat,node_memory,node_asr,node_vision,node_autonomy,node_dream toneAmber
class node_voice,node_song,node_duet,node_modelstation toneMint
class node_tuner,node_admin toneRose
class node_cluster,node_obs,node_llm,node_speech_services toneIndigo
```

## Components

### Desktop experience

- [Pet and management UI](https://github.com/nbllt666/cx-o/tree/main/APP-Frontend/src): `APP-Frontend/src`
- [Electron host](https://github.com/nbllt666/cx-o/blob/main/APP-Frontend/electron/main.ts): `APP-Frontend/electron/main.ts`
- [Live2D and VRM](https://github.com/nbllt666/cx-o/tree/main/APP-Frontend/src/avatar): `APP-Frontend/src/avatar`
- [Screen perception](https://github.com/nbllt666/cx-o/blob/main/APP-Frontend/src/hooks/useVisionPipeline.ts): `APP-Frontend/src/hooks/useVisionPipeline.ts`
- [Meeting experience](https://github.com/nbllt666/cx-o/blob/main/APP-Frontend/src/pages/pet/MeetingRoomView.tsx): `APP-Frontend/src/pages/pet/MeetingRoomView.tsx`

### Companion backend

- [HTTP and WebSocket API](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/api): `CX-O-SERVER/server/api`
- [Conversation and tools](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/chat): `CX-O-SERVER/server/core/chat`
- [Memory and retrieval](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/memory): `CX-O-SERVER/server/core/memory`
- [Speech recognition](https://github.com/nbllt666/cx-o/blob/main/CX-O-SERVER/asr_container/streaming_engine.py): `CX-O-SERVER/asr_container/streaming_engine.py`
- [Vision processing](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/vision): `CX-O-SERVER/server/core/vision`
- [Autonomous life](https://github.com/nbllt666/cx-o/blob/main/CX-O-SERVER/server/autonomy/core/loop/autonomy_engine.py): `CX-O-SERVER/server/autonomy/core/loop/autonomy_engine.py`
- [Dream engine](https://github.com/nbllt666/cx-o/blob/main/CX-O-SERVER/server/autonomy/dream/engine.py): `CX-O-SERVER/server/autonomy/dream/engine.py`

### Voice and music

- [Voice workstation](https://github.com/nbllt666/cx-o/tree/main/CX-O-VoiceWorkStation/workstation/services): `CX-O-VoiceWorkStation/workstation/services`
- [Song generation](https://github.com/nbllt666/cx-o/blob/main/CX-O-VoiceWorkStation/workstation/services/song_pipeline.py): `CX-O-VoiceWorkStation/workstation/services/song_pipeline.py`
- [Duet conversion](https://github.com/nbllt666/cx-o/blob/main/CX-O-VoiceWorkStation/workstation/services/duet_pipeline.py): `CX-O-VoiceWorkStation/workstation/services/duet_pipeline.py`
- [Model training](https://github.com/nbllt666/cx-o/blob/main/CXO-ModelStation/modelstation/main.py): `CXO-ModelStation/modelstation/main.py`

### Management and evolution

- [Evolution training](https://github.com/nbllt666/cx-o/blob/main/CXO-Tuner/tuner/api/routes.py): `CXO-Tuner/tuner/api/routes.py`
- [CX-A management](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/admin): `CX-O-SERVER/server/core/admin`

### Integrations and resilience

- [Sentinel cluster](https://github.com/nbllt666/cx-o/tree/main/CX-O-SERVER/server/core/cluster): `CX-O-SERVER/server/core/cluster`

### Other components

- **User**
- **OBS**
- **Ollama and LLMs**
- **ASR and TTS services**

## Connections

- User → Pet and management UI: interacts (evidence: `README.md`)
- Pet and management UI → HTTP and WebSocket API: sends chat (evidence: `README.md`)
- HTTP and WebSocket API → Conversation and tools: routes requests
- Conversation and tools → Memory and retrieval: retrieves memories (evidence: `README.md`)
- Conversation and tools → Ollama and LLMs: requests generation (evidence: `README.md`)
- HTTP and WebSocket API → Speech recognition: uses speech recognition (evidence: `README.md`)
- Pet and management UI → ASR and TTS services: uses speech services (evidence: `README.md`)
- Pet and management UI → Live2D and VRM: renders character (evidence: `README.md`)
- Pet and management UI → Screen perception: uploads captured clips (evidence: `APP-Frontend/src/hooks/useVisionPipeline.ts`)
- Screen perception → HTTP and WebSocket API: uploads perception data
- HTTP and WebSocket API → Vision processing: processes screen input (evidence: `README.md`)
- Pet and management UI → Meeting experience: starts meetings (evidence: `README.md`)
- Meeting experience → HTTP and WebSocket API: connects to meeting service (evidence: `README.md`)
- HTTP and WebSocket API → Autonomous life: runs autonomous cycles (evidence: `README.md`)
- Autonomous life → Dream engine: starts dream sessions
- Pet and management UI → Voice workstation: requests voice work (evidence: `README.md`)
- Voice workstation → Song generation: generates songs (evidence: `CX-O-VoiceWorkStation/workstation/services/song_pipeline.py`)
- Voice workstation → Duet conversion: converts duet vocals (evidence: `CX-O-VoiceWorkStation/workstation/services/duet_pipeline.py`)
- Pet and management UI → Model training: manages voice models (evidence: `README.md`)
- Pet and management UI → Evolution training: submits evolution feedback (evidence: `README.md`)
- CX-A management → Sentinel cluster: orchestrates nodes (evidence: `README.md`)
- HTTP and WebSocket API → Sentinel cluster: replicates state (evidence: `README.md`)
- Pet and management UI → OBS: provides streaming sources (evidence: `README.md`)
