# w-okada/voice-changer architecture

> Architecture diagram of the GitHub repository w-okada/voice-changer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/w-okada/voice-changer
- Repository: https://github.com/w-okada/voice-changer

## Overview

##### Project type and purpose
- This is a full-stack, multi-platform real-time voice conversion application (VCClient) with both local (Web Audio API/worklet) and network-offloaded (client↔server) modes. It includes:
- A web/desktop client (React+TypeScript, Webpack) under client/, a shared TypeScript library for audio worklet under client/lib/, and a minimal Python client under client/python/.
- A Python server (Flask/Socket.IO) under server/ exposing REST endpoints (server/restapi/) and WebSocket audio‐streaming (server/sio/) that dispatches audio data to a plugin-style “Voice Conversion Engine” supporting multiple model types (RVC, DiffusionSVC, EasyVC, etc.).
- A standalone recorder UI (React+TS) under recorder/.
- Trainer components and scripts under trainer/ & docker_trainer/ for model training.
- Docker setups (docker/, docker_vcclient/, docker_trainer/) and helper scripts under script/.
- Documentation (docs/, tutorials/) and CI workflows (.github/).

##### File structure
- Top-level architectural slices:
- client/ (UI and audio pipeline in browser/desktop)
- server/ (backend services: REST API, Socket.IO, voice-changer modules)
- recorder/ (corpus recording front end)
- trainer/ + docker_trainer/ (model training pipelines)
- docker*/, script/ (containerization & deployment)
- docs/, tutorials/ (user/developer docs)
- Key patterns:
- Component-based React frontends (client/demo/src/components, recorder/src/100_components)
- Shared TypeScript library for audio handling & worklet nodes (client/lib/)
- Plugin-style server modules in server/voice_changer/, dynamically loaded by VoiceChangerManager
- Dual communication paths: HTTP/REST for metadata and file upload/download; WebSocket (Socket.IO) for real-time audio streams
- Docker images per role (vcclient, trainer, vcclient network mode)

##### README for additional insights
- The README emphasizes:
- Multi-AI-model support (RVC, Beatrice v1/v2, MMVC, etc.) via plugin modules.
- Standalone (local) and network modes (offload CPU/GPU to server)
- Cross-platform support (Windows/Mac/Linux/Colab)
- A REST API for programmatic control and a Socket.IO layer for streaming.

a. Main components
1. Client UI (React/TS browser or Electron desktop)
2. Client audio pipeline (Web Audio API → VoiceChangerWorkletNode)
3. Client networking layer (REST client, Socket.IO client)
4. Server API layer (Flask-based REST endpoints + MMVC_Rest etc.)
5. Server real-time stream layer (Socket.IO namespace under server/sio/)
6. Voice Conversion Engine
- Plugin registry loading modules under server/voice_changer/
- Individual modules for each AI model (RVC, DiffusionSVC, etc.)
7. Model repository/storage (server/data/, server/model_dir_static/)
8. Trainer pipeline (PyTorch/ONNX conversion under trainer/, docker_trainer/)
9. Recorder UI (separate React app under recorder/)
10. Docker containers (vcclient, vcclient network mode, trainer)
11. Documentation portal (docs/)

b. Relationships & interactions
- UI triggers mic capture → feeds into Web Audio worklet → (local mode) worklet processes audio via client/lib → output to speakers
- (Network mode) worklet sends PCM chunks over Socket.IO to the server stream namespace
- Server receives audio chunks → VoiceChangerManager dispatches to the correct model plugin → processed chunks streamed back over Socket.IO → client plays audio
- REST endpoints used for: model listing/download (MMVC_Rest_Fileuploader), slot management, server info, file uploads, error handling
- Trainer pipelines triggered separately in dedicated containers, produce new model artifacts vs. onnx exporters

c. Architectural patterns/design principles
- Micro-frontend plus micro-library: separate demo, recorder, and Python clients, sharing core audio-pipeline TS library
- Plugin architecture on server for voice conversion algorithms
- Dual-mode operation: local worklet vs. network offload
- Containerization per role, CI/CD workflows in GitHub Actions

d. Key technologies/frameworks
- Frontend: React, TypeScript, Webpack, PostCSS
- Audio: Web Audio API, AudioWorkletNode
- Networking: Axios/Fetch for REST, Socket.IO for streaming
- Backend: Python, Flask (or equivalent), python-socketio
- ML frameworks: PyTorch, ONNX, custom graph inference
- Docker, shell scripts for deployment/training

- Use clear labels for each component (e.g., “VCClient UI”, “VoiceChangerWorkletNode”, “Socket.IO Stream” etc.)
- Audio data flow (mic → worklet → network → server engine → network → worklet → speakers)
- Control/metadata flow (UI → REST → server; server → REST → UI)
- UI components (blue)
- Audio pipeline (green)
- Networking (orange)
- Server backend (red)
- Storage & training (gray)
- Frontend apps (rounded rectangles)
- Backend services/APIs (rectangles)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        UI["VCClient UI"]:::frontend
        SharedLib["Audio Pipeline Library"]:::audio
        RESTClient["REST & Socket.IO Client"]:::network
        PythonClient["Minimal Python Client"]:::frontend
    end

    %% Server Layer
    subgraph "Server Layer"
        direction TB
        subgraph "API Gateway (REST)" 
            direction TB
            REST_Hello["Hello Endpoint"]:::backend
            REST_VoiceChanger["VoiceChanger Endpoint"]:::backend
            REST_FileUploader["File Uploader"]:::backend
            REST_Core["Core REST API"]:::backend
        end
        subgraph "Stream Gateway (Socket.IO)"
            direction TB
            SIO_App["SocketIO App"]:::backend
            SIO_Server["SocketIOServer"]:::backend
            SIO_Namespace["Audio Namespace"]:::backend
        end
        subgraph "Voice Conversion Engine"
            direction TB
            VC_Manager["VoiceChangerManager"]:::backend
            VC_Core["VoiceChanger Core"]:::backend
            subgraph "Model Plugins"
                direction TB
                RVC["RVC"]:::external
                Diffusion["DiffusionSVC"]:::external
                EasyVC["EasyVC"]:::external
                SoVits["SoVitsSvc40"]:::external
                MMVCv13["MMVCv13"]:::external
                MMVCv15["MMVCv15"]:::external
                LLVC["LLVC"]:::external
                Beatrice["Beatrice"]:::external
                DDSP["DDSP_SVC"]:::external
            end
        end
        subgraph "Storage"
            direction TB
            ModelRepo["Model Repository"]:::storage
            TempStorage["Temporary Storage"]:::storage
        end
    end

    %% Other Components
    subgraph "Recorder App"
        direction TB
        RecorderUI["Recorder UI"]:::frontend
    end

    subgraph "Trainer Subsystem"
        direction TB
        Trainer["Trainer Pipelines"]:::external
        DockerTrainer["Docker Trainer"]:::external
    end

    subgraph "Containerization & Scripts"
        direction TB
        DockerClient["Docker VCClient"]:::external
        DockerTrainerSetup["Docker Trainer Setup"]:::external
        Scripts["Helper Scripts"]:::external
    end

    subgraph "Documentation"
        direction TB
        Docs["Docs & Tutorials"]:::external
    end

    %% Relationships
    UI --> SharedLib
    SharedLib --> UI
    SharedLib --> RESTClient
    RESTClient -->|"REST Calls"| REST_Core
    RESTClient -->|"Socket.IO Streams"| SIO_Namespace
    PythonClient -->|"REST & Socket.IO"| REST_Core

    SIO_Namespace --> VC_Manager
    REST_Core --> VC_Manager
    VC_Manager --> VC_Core
    VC_Manager --> ModelRepo
    VC_Core --> ModelRepo
    VC_Core --> TempStorage
    VC_Manager --> RVC
    VC_Manager --> Diffusion
    VC_Manager --> EasyVC
    VC_Manager --> SoVits
    VC_Manager --> MMVCv13
    VC_Manager --> MMVCv15
    VC_Manager --> LLVC
    VC_Manager --> Beatrice
    VC_Manager --> DDSP

    RecorderUI --> SharedLib

    Trainer --> ModelRepo
    DockerTrainer --> Trainer

    DockerClient -.-> UI
    DockerClient -.-> PythonClient
    DockerClient -.-> RESTClient

    Scripts -.-> DockerClient
    Scripts -.-> DockerTrainer

    Docs -.-> UI
    Docs -.-> Trainer
    Docs -.-> REST_Core

    %% Click Events
    click UI "https://github.com/w-okada/voice-changer/tree/master/client/demo/src"
    click SharedLib "https://github.com/w-okada/voice-changer/tree/master/client/lib/src"
    click SharedLib "https://github.com/w-okada/voice-changer/tree/master/client/lib/worklet/src"
    click RESTClient "https://github.com/w-okada/voice-changer/blob/master/client/lib/src/client/ServerRestClient.ts"
    click RESTClient "https://github.com/w-okada/voice-changer/blob/master/client/lib/src/client/ServerConfigurator.ts"
    click PythonClient "https://github.com/w-okada/voice-changer/blob/master/client/python/vc_client.py"
    click REST_Core "https://github.com/w-okada/voice-changer/blob/master/server/restapi/MMVC_Rest.py"
    click REST_FileUploader "https://github.com/w-okada/voice-changer/blob/master/server/restapi/MMVC_Rest_Fileuploader.py"
    click REST_VoiceChanger "https://github.com/w-okada/voice-changer/blob/master/server/restapi/MMVC_Rest_VoiceChanger.py"
    click REST_Hello "https://github.com/w-okada/voice-changer/blob/master/server/restapi/MMVC_Rest_Hello.py"
    click SIO_App "https://github.com/w-okada/voice-changer/blob/master/server/sio/MMVC_SocketIOApp.py"
    click SIO_Server "https://github.com/w-okada/voice-changer/blob/master/server/sio/MMVC_SocketIOServer.py"
    click SIO_Namespace "https://github.com/w-okada/voice-changer/blob/master/server/sio/MMVC_Namespace.py"
    click VC_Manager "https://github.com/w-okada/voice-changer/blob/master/server/voice_changer/VoiceChangerManager.py"
    click VC_Core "https://github.com/w-okada/voice-changer/blob/master/server/voice_changer/VoiceChanger.py"
    click RVC "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/RVC/"
    click Diffusion "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/DiffusionSVC/"
    click EasyVC "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/EasyVC/"
    click SoVits "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/SoVitsSvc40/"
    click MMVCv13 "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/MMVCv13/"
    click MMVCv15 "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/MMVCv15/"
    click LLVC "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/LLVC/"
    click Beatrice "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/Beatrice/"
    click DDSP "https://github.com/w-okada/voice-changer/tree/master/server/voice_changer/DDSP_SVC/"
    click ModelRepo "https://github.com/w-okada/voice-changer/tree/master/server/model_dir_static"
    click TempStorage "https://github.com/w-okada/voice-changer/tree/master/server/tmp_dir"
    click Trainer "https://github.com/w-okada/voice-changer/tree/master/trainer/"
    click DockerTrainer "https://github.com/w-okada/voice-changer/tree/master/docker_trainer/"
    click RecorderUI "https://github.com/w-okada/voice-changer/tree/master/recorder/src"
    click RecorderUI "https://github.com/w-okada/voice-changer/tree/master/recorder/public"
    click DockerClient "https://github.com/w-okada/voice-changer/tree/master/docker_vcclient/"
    click DockerTrainerSetup "https://github.com/w-okada/voice-changer/tree/master/docker_trainer/"
    click Scripts "https://github.com/w-okada/voice-changer/blob/master/script/001_pushDocker*.sh"
    click Docs "https://github.com/w-okada/voice-changer/tree/master/docs/"
    click Docs "https://github.com/w-okada/voice-changer/tree/master/tutorials/"

    %% Styles
    classDef frontend fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef audio fill:#d4edda,stroke:#333,stroke-width:1px;
    classDef network fill:#fff3cd,stroke:#333,stroke-width:1px;
    classDef backend fill:#f8d7da,stroke:#333,stroke-width:1px;
    classDef storage fill:#e2e3e5,stroke:#333,stroke-width:1px;
    classDef external fill:#d1ecf1,stroke:#333,stroke-width:1px;
```
