How nurdaulet-no/jarvis-ai-assistent works
Project type and purpose
• This is an asynchronous, standalone Python voice-assistant application (“Jarvis Assistant”) that captures audio from the microphone, performs voice activity detection (VAD), converts speech to text, sends the text to an AI “brain” (Google Gemini), receives a response, and synthesizes speech to play back through speakers.
• Key use cases: real-time voice interaction, customizable AI personality, CPU-only execution, asynchronous pipeline.
File structure
• Top-level scripts:
• main.py — entry point; orchestrates the async event loop and ties all modules together
• test.py — quick test harness
• Directories:
• config/ — application configuration (settings.py)
• core/ — core functional modules:
– ai_core.py — handles API calls to Google Gemini
– audio_handler.py — captures/resamples audio (sounddevice, torchaudio, numpy)
– speech_recognizer.py — performs STT via faster-whisper
– speech_synthesizer.py — performs TTS via Coqui TTS (xtts_v2) and voice cloning
• utils/ — helper functions (logging, error handling, prompt management)
• resources/ — sample audio files for demo
• Configuration files: requirements.txt, .gitignore, LICENSE
README for architecture insights
• Asynchronous processing throughout (asyncio)
• VAD integration via Silero VAD for smart listening
• Splitting long AI responses to meet TTS limits
• AI personality customization via system prompts
a. Main components
• Microphone Input
• Audio Handler (resampling)
• VAD Module
• Speech Recognizer (STT)
• AI Core (Google Gemini API)
• Response Splitter (to chunk long replies)
• Speech Synthesizer (TTS)
• Speaker Output
• Configuration & Utilities (cross-cutting)
• Async Event Loop (orchestrator)
b. Relationships and interactions
1. Microphone → Audio Handler
2. Audio Handler → VAD (if voice detected) → Speech Recognizer
3. STT output → AI Core → receives text response
4. AI response → Response Splitter → Speech Synthesizer
5. TTS audio → Speaker Output
c. Architectural patterns/design principles
• Pipeline pattern: chained async tasks for audio capture, detection, recognition, AI call, synthesis, playback
• Observer/Callback pattern: VAD triggers STT only on speech segments
• Cross-cutting config–utility modules injected or imported by core modules for settings, logging, prompts
d. Key technologies
• asyncio for concurrency
• faster-whisper for STT
• Silero VAD for voice detection
• Google Gemini API for the “brain”
• Coqui TTS (xtts_v2) for speech synthesis
• sounddevice, torchaudio, numpy for audio I/O and processing