How vnktadithya/platform-agnostic-voice-first-music-assistant works
SAM is a voice- and text-controlled music assistant: users connect a music platform, submit a request, and receive a conversational response while SAM interprets intent, resolves context, and controls playback. The map preserves the documented Spotify/SoundCloud integrations, search, library synchronization, audio ducking, and reactive 3D interface. Sampled code confirms the chat/voice orchestration, deferred actions, account connection, and Redis-backed session state; unsampled subsystem wiring is shown only where README or the tree establishes the capability, not as speculative call chains.
Main entry points: useChatController.ts Chat controller · dialog_manager.py Dialog manager · chat_routes.py Chat routes · adapter_factory.py Adapter factory
Components 29
Client experience
- App routes
frontend/src/App.tsx - Chat overlay
frontend/src/pages/ChatOverlay.tsx - Chat controller
frontend/src/hooks/useChatController.ts - Chat handler
frontend/src/hooks/useChatHandler.ts - Voice client
frontend/src/services/VoiceClient.ts - Audio ducking
frontend/src/hooks/useAudioController.ts - API client
frontend/src/services/api.ts - 3D interface
frontend/src/components/canvas/SamCore.tsx
API and identity
- FastAPI app
backend/main.py - Chat routes
backend/api/v1/chat_routes.py - Platform login
backend/api/v1/adapter_routes.py - Live state channel
backend/socket_manager.py
Conversation runtime
- Dialog manager
backend/services/dialog_manager.py - Intent and actions
backend/services/LLM_service.py - Speech recognition
backend/services/speech_to_text.py - Speech synthesis
backend/services/text_to_speech.py
Music platforms
- Music actions
backend/services/music_action_service.py - Adapter factory
backend/adapters/adapter_factory.py - Spotify adapter
backend/adapters/spotify_adapter.py - SoundCloud adapter
backend/adapters/soundcloud_adapter.py - Music search
backend/api/v1/search_routes.py
State and sync
- Library sync
backend/services/library_sync_service.py - Background worker
backend/celery_worker.py - Session manager
backend/services/session_manager.py - Redis
backend/configurations/redis_client.py - Account database
backend/configurations/database.py
Other components
- Listener
- Groq LLM
- Music services
Connections 38
- Listener → App routes: opens
- App routes → Chat overlay: routes to
- Chat overlay → Chat controller: uses
- Chat controller → Chat handler: delegates input
- Chat controller → Voice client: records voice
- Chat handler → Voice client: stops recording
- Chat handler → API client: sends chat
- Chat controller → Audio ducking: ducks audio
- Audio ducking → API client: requests volume
- API client → FastAPI app: HTTP requests
- FastAPI app → Chat routes: mounts routes
- FastAPI app → Platform login: mounts routes
- FastAPI app → Live state channel: mounts channel
- Chat routes → Speech recognition: transcribes audio
- Chat routes → Dialog manager: processes request
- Chat routes → Dialog manager: dispatches action
- Dialog manager → Session manager: reads and updates
- Session manager → Redis: stores state
- Dialog manager → Intent and actions: requests interpretation
- Intent and actions → Groq LLM: sends prompt
- Dialog manager → Speech synthesis: synthesizes reply
- Chat routes → Chat controller: returns response
- Chat controller → API client: executes deferred action
- Dialog manager → Music actions: executes intent
- Music actions → Adapter factory: creates adapter
- Adapter factory → Spotify adapter: selects
- Adapter factory → SoundCloud adapter: selects
- Spotify adapter → Music services: controls playback
- SoundCloud adapter → Music services: controls playback
- Platform login → Spotify adapter: authenticates
- Platform login → SoundCloud adapter: authenticates
- Platform login → Account database: saves accounts
- Music search → Adapter factory: searches providers
- Background worker → Library sync: schedules sync
- Library sync → Adapter factory: reads libraries
- Library sync → Account database: updates library data
- Live state channel → Listener: emits state
- Chat controller → 3D interface: drives experience