# run-llama/voice-chat-pdf architecture

> Architecture diagram of the GitHub repository run-llama/voice-chat-pdf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/run-llama/voice-chat-pdf
- Repository: https://github.com/run-llama/voice-chat-pdf

## Overview

Key Components to Include:

##### Frontend Layer
- Main components:
- Console Interface (ConsolePage)
- Component
- Voice Input Controls (Toggle/Button components)
- Use React component symbols

##### Client-Side Processing
- WavTools Library (show components for recording, streaming, analysis)
- Audio Worklets (stream_processor, audio_processor)

##### Backend Services
- Next.js API Routes layer
- LlamaIndex RAG System:
- Document Processing
- Embedding Generation
- Query Engine
- Relay Server (separate service)

##### External Services
- OpenAI API (show connection for real-time API)
- Document Storage (PDF files)

- [User/Browser Layer]
- ↕
- [Client Processing Layer]
- ↕
- [Backend Services Layer]
- ↕
- [External Services Layer]

##### Connection Types
- Double lines: Streaming connections

Specific Implementation Details:

##### Frontend Section
- [Next.js Frontend Container]
- Console Interface
- Voice Controls
- Map Component

##### Audio Processing
- [WavTools Library]
- Recorder
- Stream Player
- Audio Analysis

##### Backend Services
- [Next.js API Routes]
- LlamaIndex RAG
- Engine Controllers

Important Elements to Highlight:

##### Real-time Communication Flow
- WebSocket connections for real-time voice
- Streaming responses from OpenAI
- Bidirectional voice communication

##### RAG Pipeline
- Document ingestion flow
- Embedding generation process
- Query processing path

##### State Management
- Voice processing states
- Conversation context flow
- Document context handling

Additional Considerations:

##### Notes for
- Authentication points (API key requirements)
- Voice processing modes (manual/VAD)
- Document processing pipeline

##### System Requirements
- Browser compatibility
- Microphone access
- API key configuration

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Browser
        FE[Next.js Frontend]
        CI["Console Interface"]:::frontend
        VC["Voice Controls"]:::frontend
        MC["Map Component"]:::frontend
        FE --> CI
        FE --> VC
        FE --> MC
    end

    subgraph "Audio Processing"
        WL["WavTools Library"]:::audio
        AP["Audio Processor"]:::audio
        SP["Stream Processor"]:::audio
        AA["Audio Analysis"]:::audio
        WL --> AP
        WL --> SP
        WL --> AA
    end

    subgraph "Backend Services"
        API["Next.js API Routes"]:::backend
        RAG["LlamaIndex RAG System"]:::backend
        RS["Relay Server"]:::backend
        CM["Context Management"]:::backend
        EC["Engine Controllers"]:::backend
        
        API --> RAG
        API --> RS
        API --> CM
        RAG --> EC
    end

    subgraph "External Services"
        OAPI["OpenAI API"]:::external
        DS["Document Storage"]:::external
    end

    %% Connections
    Browser <--> |WebSocket| RS
    Browser <--> |HTTP| API
    RS <--> OAPI
    RAG <--> DS
    Browser <--> |Audio Stream| WL

    %% Click Events
    click CI "https://github.com/run-llama/voice-chat-pdf/blob/main/src/pages/ConsolePage.tsx"
    click MC "https://github.com/run-llama/voice-chat-pdf/blob/main/src/components/MapComponent.tsx"
    click VC "https://github.com/run-llama/voice-chat-pdf/blob/main/src/components/button/Button.tsx"
    click WL "https://github.com/run-llama/voice-chat-pdf/tree/main/src/lib/wavtools/"
    click AP "https://github.com/run-llama/voice-chat-pdf/blob/main/src/lib/wavtools/lib/worklets/audio_processor.js"
    click SP "https://github.com/run-llama/voice-chat-pdf/blob/main/src/lib/wavtools/lib/worklets/stream_processor.js"
    click AA "https://github.com/run-llama/voice-chat-pdf/blob/main/src/lib/wavtools/lib/analysis/audio_analysis.js"
    click RAG "https://github.com/run-llama/voice-chat-pdf/blob/main/src/pages/engine/generate.ts"
    click CM "https://github.com/run-llama/voice-chat-pdf/blob/main/src/pages/api/context.ts"
    click RS "https://github.com/run-llama/voice-chat-pdf/blob/main/relay-server/lib/relay.js"
    click DS "https://github.com/run-llama/voice-chat-pdf/tree/main/data/"

    %% Styling
    classDef frontend fill:#3498db,stroke:#2980b9,color:white
    classDef audio fill:#2ecc71,stroke:#27ae60,color:white
    classDef backend fill:#f1c40f,stroke:#f39c12,color:black
    classDef external fill:#9b59b6,stroke:#8e44ad,color:white

    %% Legend
    subgraph Legend
        L1["Frontend Components"]:::frontend
        L2["Audio Processing"]:::audio
        L3["Backend Services"]:::backend
        L4["External Services"]:::external
    end
```
