How aws-samples/sample-nova-sonic-agentic-chatbot works
Project Type and Purpose
• This is a full‐stack demo application combining a Next.js/TypeScript/Tailwind CSS frontend with a FastAPI/Python backend.
• Purpose: Showcase Nova Sonic’s modular tool system, real‐time audio capture/playback, and dynamic UI rendering for dual results (model context + UI payload).
Top‐Level Components
• Frontend (frontend/)
• Next.js App Router (app/), React components (components/), global styles, config, and utility functions.
• UI components broken out into:
– Apps (e.g. EchoApp)
– Tool‐Outputs for each UI result type (Text, Card, Image, Video, PDF…)
– Base UI primitives (Button, Slider, Card…)
• WebSocket client + audio capture via MediaRecorder APIs.
• Backend (backend/)
• FastAPI application (main.py), WebSocket endpoint for voice sessions.
• api/apps/echo.py sample endpoint.
• Nova Sonic integration (nova_sonic_simple.py) that drives conversation & delegates to tools.
• Modular Tool System:
– Base classes & registry (tools/base/)
– Category folders (tools/categories/{utility,media,order})
– Tool Manager (tool_manager.py) that discovers and registers tools.
• Optional debug audio storage (backend/debug_audio/).
• External/Infrastructure
• Nova Sonic LLM or voice‐AI service (via nova_sonic_simple.py).
• Client browser.
• Local filesystem storage (debug audio).
Key Relationships & Data Flows
1. User interacts in browser → clicks Power → WebSocket connection established to FastAPI.
2. Click Mic → MediaRecorder starts streaming audio chunks over WebSocket.
3. FastAPI receives audio stream → passes to Nova Sonic service wrapper.
4. Nova Sonic orchestrates tool invocation:
• Tool Manager selects appropriate tool(s) from registry.
• Each tool execute() returns { model_result, ui_result }.
5. Backend merges results → streams dual‐payload back to frontend over WebSocket.
6. Frontend receives ui_result.type → renders using appropriate React component.
7. If SAVE_DEBUG_AUDIO=true, raw audio saved to backend/debug_audio/.
Architectural Patterns & Design Principles
• Separation of Concerns: UI vs API vs Tool logic.
• Plugin pattern: Tools register via a central registry.
• Async non‐blocking pipelines: FastAPI + async tool execution.
• Dual‐result pattern: decouple model context from UI rendering payload.
• Component‐based UI: React + Next.js for dynamic rendering based on ui_result.type.
• Browser (Client)
• UI Layer: React/Next.js, components, MediaRecorder, WebSocket client.
• Frontend Server (Next.js)
• App Router, Static Assets, WebSocket proxy (if used).
• Backend Server (FastAPI)
• WebSocket endpoint
• HTTP API endpoints
• Nova Sonic service integration
• Tool Manager & Tool Registry
• Tool Categories (Utility, Media, Order)
• Debug Audio Storage
• External Service
• Nova Sonic LLM/Voice Engine
• Data Store (optional)
• Filesystem for debug audio.