# parthivshah33/realtime-rag-voicebot architecture

> Architecture diagram of the GitHub repository parthivshah33/realtime-rag-voicebot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/parthivshah33/realtime-rag-voicebot
- Repository: https://github.com/parthivshah33/realtime-rag-voicebot

## Overview

##### Project type and purpose
- This is a Python‐based voice assistant (“voicebot”) proof of concept. It captures live audio, converts speech to text, processes it (optionally with retrieval-augmented generation), and returns spoken answers via text-to-speech. The “experiments” folder contains notebooks and scripts for exploring different STT/TTS engines, embedding generation, RAG, and LangChain memory.

##### File structure
- Top level:
- voicebot.py: main orchestration script
- logger.py: centralized logging
- .env + requirements.txt: configuration and dependencies
- utils/: core modules for STT, TTS, embeddings, RAG processing
- experiments/: exploratory notebooks and scripts for different speech/LLM pipelines
- knowledge documents/: static PDFs used as retrieval corpus
- Patterns: single-process pipeline, modular handlers for each stage (STT → LLM → TTS), external API reliance (Deepgram, OpenAI, etc.).

##### README for additional insights
- Basic setup steps: virtualenv, install deps, add API keys, run voicebot.py.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Core Pipeline
    Microphone["User Device / Microphone"]:::internal
    STT["STT Module"]:::internal
    Orchestrator["Orchestration Engine"]:::internal
    Logger["Logging Component"]:::internal
    Config["Configuration Store"]:::static
    Embedding["Embedding Generator"]:::internal
    Retrieval["Retrieval Module"]:::internal
    Memory["LangChain Memory"]:::internal
    LLM["LLM Service (OpenAI API)"]:::external
    TTS["TTS Module"]:::internal
    ExternalTTS["TTS Service"]:::external
    Knowledge["Knowledge Store"]:::static
    Speaker["Speakers / Audio Output"]:::internal

    %% Data Flows
    Microphone -->|"audio stream"| STT
    STT -->|"transcript"| Orchestrator
    Config --> Orchestrator
    Orchestrator --> Logger
    Orchestrator -->|"text input"| Embedding
    Embedding -->|"vectors"| Orchestrator
    Orchestrator -->|"query"| Retrieval
    Retrieval --> Knowledge
    Orchestrator --> Memory
    Memory --> LLM
    Orchestrator -->|"prompt text"| LLM
    LLM -->|"response text"| Orchestrator
    Orchestrator -->|"response text"| TTS
    TTS -->|"audio stream"| ExternalTTS
    ExternalTTS --> Speaker

    %% Experiments Integration
    subgraph "Experiments"
        EP_STT1["speech_to_text.ipynb"]:::internal
        EP_STT2["speech_to_text_streaming.py"]:::internal
        EP_STT3["gradio_audio_streamin.py"]:::internal
        EP_TTS1["text_to_speech.ipynb"]:::internal
        EP_TTS2["text_to_speech_exp.py"]:::internal
        EP_TTS3["text_to_speech_main.py"]:::internal
        EP_TTS4["streaming_text_to_speech.py"]:::internal
        EP_TTS5["deepgram_TTS.py"]:::internal
        EP_EMB1["embedding_generation.ipynb"]:::internal
        EP_RAG1["rag_experiments.ipynb"]:::internal
        EP_MEM1["langchain_memory.ipynb"]:::internal
    end

    EP_STT1 --> STT
    EP_STT2 --> STT
    EP_STT3 --> STT
    EP_TTS1 --> TTS
    EP_TTS2 --> TTS
    EP_TTS3 --> TTS
    EP_TTS4 --> TTS
    EP_TTS5 --> TTS
    EP_EMB1 --> Embedding
    EP_RAG1 --> Retrieval
    EP_MEM1 --> Memory

    %% Click Events
    click STT "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/utils/speech_to_text.py"
    click TTS "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/utils/text_to_speech.py"
    click Embedding "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/utils/embeddings.py"
    click Retrieval "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/utils/process_rag.py"
    click Knowledge "https://github.com/parthivshah33/realtime-rag-voicebot/tree/main/knowledge documents/"
    click Orchestrator "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/voicebot.py"
    click Logger "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/logger.py"
    click Config "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/.env"
    click EP_STT1 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/speech_to_text.ipynb"
    click EP_STT2 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/speech_to_text_streaming.py"
    click EP_STT3 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/live_speech_to_text/gradio_audio_streamin.py"
    click EP_TTS1 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/text_to_speech.ipynb"
    click EP_TTS2 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/text_to_speech_exp.py"
    click EP_TTS3 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/text_to_speech_main.py"
    click EP_TTS4 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/live_text_to_speech/streaming_text_to_speech.py"
    click EP_TTS5 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/deepgram_TTS.py"
    click EP_EMB1 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/embedding_generation.ipynb"
    click EP_RAG1 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/rag_experiments.ipynb"
    click EP_MEM1 "https://github.com/parthivshah33/realtime-rag-voicebot/blob/main/experiments/langchain_memory.ipynb"

    %% Styles
    classDef internal fill:#D6EAF8,stroke:#3498DB
    classDef external fill:#D5F5E3,stroke:#27AE60
    classDef static fill:#E5E8E8,stroke:#85929E
```
