# nurdaulet-no/jarvis-ai-assistent architecture

> Architecture diagram of the GitHub repository nurdaulet-no/jarvis-ai-assistent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nurdaulet-no/jarvis-ai-assistent
- Repository: https://github.com/nurdaulet-no/jarvis-ai-assistent

## Overview

##### Project type and purpose
- This is an asynchronous, standalone Python voice-assistant application (“Jarvis Assistant”) that captures audio from the microphone, performs voice activity detection (VAD), converts speech to text, sends the text to an AI “brain” (Google Gemini), receives a response, and synthesizes speech to play back through speakers.
- Key use cases: real-time voice interaction, customizable AI personality, CPU-only execution, asynchronous pipeline.

##### File structure
- Top-level scripts:
- main.py — entry point; orchestrates the async event loop and ties all modules together
- test.py — quick test harness
- Directories:
- config/ — application configuration (settings.py)
- core/ — core functional modules:
– ai_core.py — handles API calls to Google Gemini
– audio_handler.py — captures/resamples audio (sounddevice, torchaudio, numpy)
– speech_recognizer.py — performs STT via faster-whisper
– speech_synthesizer.py — performs TTS via Coqui TTS (xtts_v2) and voice cloning
- utils/ — helper functions (logging, error handling, prompt management)
- resources/ — sample audio files for demo
- Configuration files: requirements.txt, .gitignore, LICENSE

##### README for architecture insights
- Asynchronous processing throughout (`asyncio`)
- VAD integration via Silero VAD for smart listening
- Splitting long AI responses to meet TTS limits
- AI personality customization via system prompts

a. Main components
- Microphone Input
- Audio Handler (resampling)
- VAD Module
- Speech Recognizer (STT)
- AI Core (Google Gemini API)
- Response Splitter (to chunk long replies)
- Speech Synthesizer (TTS)
- Speaker Output
- Configuration & Utilities (cross-cutting)
- Async Event Loop (orchestrator)
b. Relationships and interactions
1. Microphone → Audio Handler
2. Audio Handler → VAD (if voice detected) → Speech Recognizer
3. STT output → AI Core → receives text response
4. AI response → Response Splitter → Speech Synthesizer
5. TTS audio → Speaker Output
c. Architectural patterns/design principles
- Pipeline pattern: chained async tasks for audio capture, detection, recognition, AI call, synthesis, playback
- Observer/Callback pattern: VAD triggers STT only on speech segments
- Cross-cutting config–utility modules injected or imported by core modules for settings, logging, prompts
d. Key technologies
- asyncio for concurrency
- faster-whisper for STT
- Silero VAD for voice detection
- Google Gemini API for the “brain”
- Coqui TTS (xtts_v2) for speech synthesis
- sounddevice, torchaudio, numpy for audio I/O and processing

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Async Event Loop"
        direction TB
        Mic["Microphone Input"]:::io
        AudioHandler["Audio Handler\n(resample & VAD)"]:::io
        VAD["VAD Module"]:::processing
        STT["Speech Recognizer\n(faster-whisper)"]:::processing
        AICore["AI Core\n(Google Gemini API)"]:::external
        Splitter["Response Splitter"]:::processing
        TTS["Speech Synthesizer\n(Coqui TTS)"]:::processing
        Speaker["Speaker Output"]:::io
    end

    Config["Configuration\n(settings)"]:::config
    Utils["Utilities\n(logging, prompts)"]:::config
    Test["Test Harness"]:::config
    Resources["Demo Audio Resources"]:::io

    Mic -->|"raw audio"| AudioHandler
    AudioHandler -->|"audio chunk"| VAD
    VAD -->|"speech segment"| STT
    STT -->|"text prompt"| AICore
    AICore -->|"text response"| Splitter
    Splitter -->|"text chunk"| TTS
    TTS -->|"audio buffer"| Speaker

    Config --> AICore
    Config --> STT
    Config --> TTS
    Config --> AudioHandler

    Utils --> AICore
    Utils --> STT
    Utils --> TTS
    Utils --> AudioHandler

    click Mic "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/audio_handler.py"
    click AudioHandler "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/audio_handler.py"
    click VAD "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/audio_handler.py"
    click STT "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/speech_recognizer.py"
    click AICore "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/ai_core.py"
    click Splitter "https://github.com/nurdaulet-no/jarvis-ai-assistent/tree/nurda/utils/"
    click TTS "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/speech_synthesizer.py"
    click Speaker "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/core/audio_handler.py"
    click Config "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/config/settings.py"
    click Utils "https://github.com/nurdaulet-no/jarvis-ai-assistent/tree/nurda/utils/"
    click Test "https://github.com/nurdaulet-no/jarvis-ai-assistent/blob/nurda/test.py"
    click Resources "https://github.com/nurdaulet-no/jarvis-ai-assistent/tree/nurda/resources/"

    classDef io fill:#e0ffe0,stroke:#2f8f2f;
    classDef processing fill:#e0f0ff,stroke:#2070bf;
    classDef external fill:#ffe0b3,stroke:#bf6000;
    classDef config fill:#f0f0f0,stroke:#888888;
```
