# vnktadithya/platform-agnostic-voice-first-music-assistant architecture

> Architecture diagram of the GitHub repository vnktadithya/platform-agnostic-voice-first-music-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vnktadithya/platform-agnostic-voice-first-music-assistant
- Repository: https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant

## Overview

SAM is a voice- and text-controlled music assistant: users connect a music platform, submit a request, and receive a conversational response while SAM interprets intent, resolves context, and controls playback. The map preserves the documented Spotify/SoundCloud integrations, search, library synchronization, audio ducking, and reactive 3D interface. Sampled code confirms the chat/voice orchestration, deferred actions, account connection, and Redis-backed session state; unsampled subsystem wiring is shown only where README or the tree establishes the capability, not as speculative call chains.

Main entry points: [`frontend/src/hooks/useChatController.ts`](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useChatController.ts) (Chat controller), [`backend/services/dialog_manager.py`](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/dialog_manager.py) (Dialog manager), [`backend/api/v1/chat_routes.py`](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/chat_routes.py) (Chat routes), [`backend/adapters/adapter_factory.py`](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/adapter_factory.py) (Adapter factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Client experience"]
  node_app["App routes<br/>[App.tsx]"]
  node_chat_ui["Chat overlay<br/>[ChatOverlay.tsx]"]
  node_chat_controller["Chat controller"]
  node_chat_handler["Chat handler<br/>[useChatHandler.ts]"]
  node_voice_client["Voice client<br/>[VoiceClient.ts]"]
  node_audio_control["Audio ducking"]
  node_api_client["API client<br/>[api.ts]"]
  node_visuals["3D interface<br/>[SamCore.tsx]"]
end

subgraph group_api["API and identity"]
  node_server["FastAPI app<br/>[main.py]"]
  node_chat_routes["Chat routes<br/>[chat_routes.py]"]
  node_adapter_routes["Platform login<br/>[adapter_routes.py]"]
  node_socket["Live state channel<br/>[socket_manager.py]"]
end

subgraph group_conversation["Conversation runtime"]
  node_dialog["Dialog manager<br/>[dialog_manager.py]"]
  node_llm["Intent and actions<br/>[LLM_service.py]"]
  node_stt["Speech recognition<br/>[speech_to_text.py]"]
  node_tts["Speech synthesis<br/>[text_to_speech.py]"]
end

subgraph group_music["Music platforms"]
  node_music_actions["Music actions"]
  node_adapter_factory["Adapter factory<br/>[adapter_factory.py]"]
  node_spotify["Spotify adapter<br/>[spotify_adapter.py]"]
  node_soundcloud["SoundCloud adapter"]
  node_search["Music search<br/>[search_routes.py]"]
end

subgraph group_state["State and sync"]
  node_sync["Library sync"]
  node_worker["Background worker<br/>[celery_worker.py]"]
  node_session["Session manager<br/>[session_manager.py]"]
  node_redis[("Redis<br/>[redis_client.py]")]
  node_database[("Account database<br/>[database.py]")]
end

node_listener(("Listener"))
node_groq["Groq LLM"]
node_music_services["Music services"]

node_listener -->|"opens"| node_app
node_app -->|"routes to"| node_chat_ui
node_chat_ui -->|"uses"| node_chat_controller
node_chat_controller -->|"delegates input"| node_chat_handler
node_chat_controller -->|"records voice"| node_voice_client
node_chat_handler -->|"stops recording"| node_voice_client
node_chat_handler -->|"sends chat"| node_api_client
node_chat_controller -->|"ducks audio"| node_audio_control
node_audio_control -->|"requests volume"| node_api_client
node_api_client -->|"HTTP requests"| node_server
node_server -->|"mounts routes"| node_chat_routes
node_server -->|"mounts routes"| node_adapter_routes
node_server -->|"mounts channel"| node_socket
node_chat_routes -->|"transcribes audio"| node_stt
node_chat_routes -->|"processes request"| node_dialog
node_chat_routes -->|"dispatches action"| node_dialog
node_dialog -->|"reads and updates"| node_session
node_session -->|"stores state"| node_redis
node_dialog -->|"requests interpretation"| node_llm
node_llm -->|"sends prompt"| node_groq
node_dialog -->|"synthesizes reply"| node_tts
node_chat_routes -->|"returns response"| node_chat_controller
node_chat_controller -->|"executes deferred action"| node_api_client
node_dialog -->|"executes intent"| node_music_actions
node_music_actions -->|"creates adapter"| node_adapter_factory
node_adapter_factory -->|"selects"| node_spotify
node_adapter_factory -->|"selects"| node_soundcloud
node_spotify -->|"controls playback"| node_music_services
node_soundcloud -->|"controls playback"| node_music_services
node_adapter_routes -->|"authenticates"| node_spotify
node_adapter_routes -->|"authenticates"| node_soundcloud
node_adapter_routes -->|"saves accounts"| node_database
node_search -.->|"searches providers"| node_adapter_factory
node_worker -->|"schedules sync"| node_sync
node_sync -.->|"reads libraries"| node_adapter_factory
node_sync -.->|"updates library data"| node_database
node_socket -->|"emits state"| node_listener
node_chat_controller -->|"drives experience"| node_visuals

click node_app "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/App.tsx"
click node_chat_ui "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/pages/ChatOverlay.tsx"
click node_chat_controller "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useChatController.ts"
click node_chat_handler "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useChatHandler.ts"
click node_voice_client "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/services/VoiceClient.ts"
click node_audio_control "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useAudioController.ts"
click node_api_client "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/services/api.ts"
click node_visuals "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/components/canvas/SamCore.tsx"
click node_server "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/main.py"
click node_chat_routes "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/chat_routes.py"
click node_adapter_routes "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/adapter_routes.py"
click node_dialog "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/dialog_manager.py"
click node_llm "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/LLM_service.py"
click node_stt "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/speech_to_text.py"
click node_tts "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/text_to_speech.py"
click node_music_actions "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/music_action_service.py"
click node_adapter_factory "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/adapter_factory.py"
click node_spotify "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/spotify_adapter.py"
click node_soundcloud "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/soundcloud_adapter.py"
click node_search "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/search_routes.py"
click node_sync "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/library_sync_service.py"
click node_worker "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/celery_worker.py"
click node_session "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/session_manager.py"
click node_redis "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/configurations/redis_client.py"
click node_database "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/configurations/database.py"
click node_socket "https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/socket_manager.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_chat_ui,node_chat_controller,node_chat_handler,node_voice_client,node_audio_control,node_api_client,node_visuals toneBlue
class node_server,node_chat_routes,node_adapter_routes,node_socket toneAmber
class node_dialog,node_llm,node_stt,node_tts toneMint
class node_music_actions,node_adapter_factory,node_spotify,node_soundcloud,node_search toneRose
class node_sync,node_worker,node_session,node_redis,node_database,node_listener,node_groq,node_music_services toneIndigo
```

## Components

### Client experience

- [App routes](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Chat overlay](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/pages/ChatOverlay.tsx): `frontend/src/pages/ChatOverlay.tsx`
- [Chat controller](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useChatController.ts): `frontend/src/hooks/useChatController.ts`
- [Chat handler](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useChatHandler.ts): `frontend/src/hooks/useChatHandler.ts`
- [Voice client](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/services/VoiceClient.ts): `frontend/src/services/VoiceClient.ts`
- [Audio ducking](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/hooks/useAudioController.ts): `frontend/src/hooks/useAudioController.ts`
- [API client](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/services/api.ts): `frontend/src/services/api.ts`
- [3D interface](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/frontend/src/components/canvas/SamCore.tsx): `frontend/src/components/canvas/SamCore.tsx`

### API and identity

- [FastAPI app](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/main.py): `backend/main.py`
- [Chat routes](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/chat_routes.py): `backend/api/v1/chat_routes.py`
- [Platform login](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/adapter_routes.py): `backend/api/v1/adapter_routes.py`
- [Live state channel](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/socket_manager.py): `backend/socket_manager.py`

### Conversation runtime

- [Dialog manager](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/dialog_manager.py): `backend/services/dialog_manager.py`
- [Intent and actions](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/LLM_service.py): `backend/services/LLM_service.py`
- [Speech recognition](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/speech_to_text.py): `backend/services/speech_to_text.py`
- [Speech synthesis](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/text_to_speech.py): `backend/services/text_to_speech.py`

### Music platforms

- [Music actions](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/music_action_service.py): `backend/services/music_action_service.py`
- [Adapter factory](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/adapter_factory.py): `backend/adapters/adapter_factory.py`
- [Spotify adapter](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/spotify_adapter.py): `backend/adapters/spotify_adapter.py`
- [SoundCloud adapter](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/adapters/soundcloud_adapter.py): `backend/adapters/soundcloud_adapter.py`
- [Music search](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/api/v1/search_routes.py): `backend/api/v1/search_routes.py`

### State and sync

- [Library sync](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/library_sync_service.py): `backend/services/library_sync_service.py`
- [Background worker](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/celery_worker.py): `backend/celery_worker.py`
- [Session manager](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/services/session_manager.py): `backend/services/session_manager.py`
- [Redis](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/configurations/redis_client.py): `backend/configurations/redis_client.py`
- [Account database](https://github.com/vnktadithya/platform-agnostic-voice-first-music-assistant/blob/main/backend/configurations/database.py): `backend/configurations/database.py`

### Other components

- **Listener**
- **Groq LLM**
- **Music services**

## Connections

- Listener → App routes: opens
- App routes → Chat overlay: routes to
- Chat overlay → Chat controller: uses
- Chat controller → Chat handler: delegates input
- Chat controller → Voice client: records voice
- Chat handler → Voice client: stops recording
- Chat handler → API client: sends chat
- Chat controller → Audio ducking: ducks audio
- Audio ducking → API client: requests volume
- API client → FastAPI app: HTTP requests
- FastAPI app → Chat routes: mounts routes
- FastAPI app → Platform login: mounts routes
- FastAPI app → Live state channel: mounts channel
- Chat routes → Speech recognition: transcribes audio
- Chat routes → Dialog manager: processes request
- Chat routes → Dialog manager: dispatches action
- Dialog manager → Session manager: reads and updates
- Session manager → Redis: stores state
- Dialog manager → Intent and actions: requests interpretation
- Intent and actions → Groq LLM: sends prompt
- Dialog manager → Speech synthesis: synthesizes reply
- Chat routes → Chat controller: returns response
- Chat controller → API client: executes deferred action
- Dialog manager → Music actions: executes intent
- Music actions → Adapter factory: creates adapter
- Adapter factory → Spotify adapter: selects
- Adapter factory → SoundCloud adapter: selects
- Spotify adapter → Music services: controls playback
- SoundCloud adapter → Music services: controls playback
- Platform login → Spotify adapter: authenticates
- Platform login → SoundCloud adapter: authenticates
- Platform login → Account database: saves accounts
- Music search → Adapter factory: searches providers
- Background worker → Library sync: schedules sync
- Library sync → Adapter factory: reads libraries
- Library sync → Account database: updates library data
- Live state channel → Listener: emits state
- Chat controller → 3D interface: drives experience