How sai-koushik-vimmadisetty/ai-confessor works
AI Confessor is a real-time voice-and-text conversational app. A user speaks or types in the browser; the frontend streams microphone audio or text to FastAPI over WebSocket. The backend transcribes audio when needed, manages per-connection conversation history, streams a response from the selected language-model provider, and optionally synthesizes speech. The browser displays streamed text and plays returned audio, with browser speech synthesis as a fallback. The README documents these capabilities; unsampled configuration and proxy behavior are represented only where their paths and roles are documented.
Main entry points: main.py FastAPI and turn handler · ChatWindow.tsx Chat interface · ws.ts WebSocket client · audio.ts Audio capture and playback
Components 19
Browser experience
- React entry
frontend/src/main.tsx - Chat interface
frontend/src/components/ChatWindow.tsx - WebSocket client
frontend/src/lib/ws.ts - Audio capture and playback
frontend/src/lib/audio.ts - PCM worklet
frontend/public/pcm-processor.js - HTTP and WS proxy
frontend/nginx.conf
Conversation backend
- FastAPI and turn handler
backend/app/main.py - Runtime settings
backend/app/config.py - Vosk transcription
backend/app/stt.py
Session state
- Session manager
backend/app/sessions.py - Conversation session
backend/app/sessions.py
AI services
- LLM provider adapter
backend/app/llm.py - Speech synthesis
backend/app/tts.py
Other components
- User
- Browser APIs
- Edge TTS
- Anthropic API
- OpenAI API
- Browser speech synthesis
Connections 24
- User → Chat interface: types or speaks
- React entry → Chat interface: renders
- Chat interface → WebSocket client: sends turns
- Chat interface → Audio capture and playback: starts capture and playback
- Audio capture and playback → Browser APIs: captures and plays
- Audio capture and playback → PCM worklet: loads processor
- Chat interface → WebSocket client: fetches config
- WebSocket client → HTTP and WS proxy: sends HTTP and WS
- HTTP and WS proxy → FastAPI and turn handler: proxies requests
- FastAPI and turn handler → WebSocket client: sends protocol events
- WebSocket client → Chat interface: delivers events
- Chat interface → Browser speech synthesis: speaks fallback text
- FastAPI and turn handler → Runtime settings: reads settings
- FastAPI and turn handler → Session manager: creates and removes sessions
- Session manager → Conversation session: creates session
- FastAPI and turn handler → Conversation session: reads and updates history
- Conversation session → FastAPI and turn handler: returns history
- FastAPI and turn handler → Vosk transcription: feeds audio and flushes transcript
- FastAPI and turn handler → LLM provider adapter: streams conversation reply
- LLM provider adapter → Anthropic API: streams completion
- LLM provider adapter → OpenAI API: streams completion
- FastAPI and turn handler → Speech synthesis: synthesizes reply
- Speech synthesis → Edge TTS: requests audio
- FastAPI and turn handler → WebSocket client: sends transcript, tokens, and audio