How mimisukemaster/ai-voicevox works
Main Components
a. Browser Client
• Pages:
• “/” (AI-討論) – index.html + client.js + styles.css
• “/question” (AI-ずんだもん) – question/index.html + question/client.js
• “/about” – about/index.html
b. Static Assets Server
• Serves everything under /public (HTML, CSS, JS, fonts, images, TtsQuestV3Voicevox.js)
c. Express Back End (server.js)
• Endpoints: GET static files, POST /api/debate, POST /api/question, etc.
• Reads .env for GEMINI_API_KEY, COHERE_API_KEY, VOICEVOX_API_KEY, plus “useLocal” flag.
• Orchestrates calls to LLMs and TTS engine.
d. External LLM Services
• Google Gemini API
• Cohere (for CommandR)
e. Voicevox TTS Engine
• Remote (Streaming): TtsQuestV3Voicevox.js module calls the streaming API endpoint
• Optional Local: Developer-run Voicevox Engine on localhost:50021
f. Deployment Platform
• Vercel (vercel.json config): server.js runs as a serverless function; public folder served as static hosting.
Relationships & Data Flows
• User interacts in browser → client.js issues XHR/Fetch to Express (serverless).
• Express receives request →
– Calls Gemini and/or Cohere API (HTTP).
– Aggregates responses, builds debate turn or answer.
– Invokes TtsQuestV3Voicevox.js to stream audio from Voicevox (remote or local).
• Express streams back audio + JSON metadata to client.
• Browser client plays audio and updates UI in real time.
Architectural Patterns
• Monolith / Single serverless function: Express handles both static and API routes.
• Proxy pattern: backend proxies browser requests to LLMs and TTS services.
• Streaming data flow: audio is streamed from TtsQuestV3Voicevox.js to client.
• Feature flag: “ローカルAPIを使う” toggles between remote Voicevox vs. local engine.
Relevant Technologies & Frameworks
• Node.js + Express
• Vercel (serverless deployment)
• @google/generative-ai, cohere-ai SDKs
• dotenv for config
• TtsQuestV3Voicevox.js (streaming TTS library)
• Plain HTML/CSS/Vanilla JS for front end
Detailed Component Breakdown (to include
• .env / vercel.json / package.json (configuration)
• server.js (Express entry point)
• public/
• index.html & client.js (Debate)
• question/index.html & client.js (Q&A)
• about/index.html
• styles.css, TtsQuestV3Voicevox.js, fonts, images
• External flows:
• HTTP POST → /api/debate
• HTTP POST → /api/question
• HTTP GET → static assets
• Conditional branch: if “localAPI” flag → call http://localhost:50021; else → call streaming API.