How aws-samples/sample-aws-strands-nova-voice-assistant works
Project type and purpose
• This is a full-stack, voice-based AI assistant.
• Frontend: React single-page app using AWS Cloudscape Design for UI and AudioWorklets for real-time audio.
• Backend: Python monolith built on AWS Strands, exposing a WebSocket server, orchestrating multi-agent collaboration, and integrating with AWS services (EC2, SSM, Backup) and AWS Bedrock (Claude 3 Haiku).
• Voice I/O via Amazon Nova Sonic (speech-to-text and text-to-speech).
File structure
• backend/src/voice_based_aws_agent/
• agents/ (supervisor + specialized agents: ec2_agent, ssm_agent, backup_agent, orchestrator)
• config/ (tool and conversation configuration)
• utils/aws_auth.py (boto3 authentication)
• utils/voice_integration/ (WebSocket server, STT/TTS session management, Nova Sonic bridge)
• main.py (entry point launching the WebSocket and agent framework)
• frontend/
• public/ (index.html, Web Manifest, AudioWorklet processor)
• src/ (React components for chat UI and event display, helper modules for audio capture, playback, and WebSocket messaging)
• Deployment scripts: run_backend.sh, run_frontend.sh
• Configuration: requirements.txt, package.json
Key insights from README
• Simplified multi-agent architecture with a Supervisor Agent routing requests to Specialized Agents (EC2, SSM, Backup).
• Real-time voice via Nova Sonic.
• Communication channel: WebSocket server in the backend, consumed by the React frontend.
• AI inference via AWS Bedrock Claude 3 Haiku model calls.
• AWS authentication split: environment variables for Nova Sonic, boto3 profiles for other services.
a. Main components (group into layers):
• User + Browser Client (UI + AudioWorklet)
• Frontend Application (React + WebSocket client)
• Backend Application
– WebSocket Server (voice_integration/server.py)
– Voice Integration Modules (session manager, STT, TTS)
– Multi-Agent Core (orchestrator, supervisor_agent)
– Specialized Agents (ec2_agent, ssm_agent, backup_agent)
– Configuration & Authentication (aws_auth, config files, tool consent)
• External Services
– Amazon Nova Sonic (speech services)
– AWS Bedrock (Claude 3 Haiku inference)
– AWS Service APIs (EC2, SSM, Backup via boto3)
b. Relationships & data flows:
1. User speaks → Browser AudioWorklet captures audio → WebSocket client
2. WebSocket message → Backend WebSocket Server
3. Server forwards raw audio to Nova Sonic STT → text transcript
4. Transcript → Supervisor Agent → invokes AWS Bedrock model for intent/understanding
5. Supervisor routes intent to the correct Specialized Agent
6. Agent calls AWS APIs (boto3) or Bedrock as needed → constructs response
7. Response text → Nova Sonic TTS → audio stream
8. Backend → sends audio chunk via WebSocket → Browser AudioWorklet plays audio
c. Architectural patterns & principles:
• Multi-agent orchestration pattern (Supervisor dispatching to Specialists)
• Single backend monolith (no separate microservices)
• Clear separation of concerns: voice I/O, agent logic, AWS integrations, UI
d. Key technologies/frameworks:
• Python 3.12+, AWS Strands, boto3
• React, AWS Cloudscape Design, AudioWorklet API
• Amazon Nova Sonic, AWS Bedrock Models, WebSocket