How voicevox/voicevox_core works
Project Type and Purpose
• This repository is the “VOICEVOX CORE” audio‐synthesis engine, written in Rust and published as a core library for speech synthesis.
• It provides a high‐performance Rust core (crates/voicevox_core), a C ABI wrapper (crates/voicevox_core_c_api), and multiple language bindings (Python, Java). Users can embed this core in downstream applications (C++, Python scripts, JVM apps, mobile runtimes).
File‐Structure Analysis
• Top‐level “crates/” directory:
• voicevox_core: the Rust engine implementing text analysis, feature extraction, inference (via ONNXRuntime), waveform generation.
• voicevox_core_c_api: exposes C functions (FFI layer) around voicevox_core for .dll/.so/.dylib distribution; uses cbindgen.
• voicevox_core_python_api: Rust-based PyO3 crate producing a wheel; wraps the C API or directly links to core.
• voicevox_core_java_api: JNI/Gradle project producing Android/iOS libraries and JVM JAR, wrapping the C API.
• voicevox_core_macros: proc macros used by core and bindings for code generation.
• test_util, downloader, xtask: helper crates for testing, model downloading, build scripting.
• Examples directory: C++ (Unix/Windows), Python, Kotlin. They consume the C API or the Python wheel.
• build_util: shell scripts for packaging iOS XCFramework and code‐signing.
• CI workflows (.github/workflows): build & test across languages, lint, release.
• model/: sample model files (VVM format) used by the engine.
README Insights
• The engine ships prebuilt C‐API libraries and Python wheels in GitHub Releases.
• The project emphasizes multi-language distribution and FFI compatibility.
• External dependencies: ONNX Runtime (for model inference), OpenJTalk (for Japanese text processing).
a. Main Components
• Rust Core (voicevox_core crate)
• Proc-Macro Helpers (voicevox_core_macros)
• C API Layer (voicevox_core_c_api)
• Python Binding (voicevox_core_python_api)
• Java Binding (voicevox_core_java_api)
• Model Downloader (crates/downloader)
• Test Utilities (crates/test_util)
• Build Scripts & Packaging (xtask, build_util)
• CI/CD Pipeline (GitHub Actions workflows)
• Example Clients (C++, Python, Kotlin projects)
b. Relationships & Data Flows
• voicevox_core depends on voicevox_core_macros and links to ONNX Runtime & OpenJTalk.
• voicevox_core_c_api links to the compiled core and exposes extern "C" functions.
• Python and Java bindings consume the C API (via FFI) or link directly to Rust via PyO3/JNI.
• Example clients call into the C API or import the Python module.
• downloader fetches model assets (.onnx, .json) into a local cache for both tests and runtime.
• xtask generates C headers, updates version info, and drives build‐time codegen.
• build_util scripts package the compiled libraries into platform‐specific frameworks.
• CI workflows build and test each crate, package artifacts, and deploy releases.
c. Architectural Patterns & Principles
• Layered FFI architecture: Core → C API → Language Bindings → Clients
• Code‐generation with proc macros and xtask for DRY FFI bindings
• Separation of concerns: model management (downloader), testing (test_util), build packaging (build_util/xtask)
• Cross-platform packaging using shell scripts and GitHub Actions
d. Key Technologies & Frameworks
• Rust and Cargo
• ONNX Runtime (inference runtime)
• OpenJTalk (text processing)
• PyO3 (Python binding)
• JNI/Gradle (Java binding)
• cbindgen (C header generation)
• GitHub Actions for CI/CD