How pennyroyaltea/gibberlink works
Project type and purpose
• This is a full-stack hackathon demo built with Next.js (App Router).
• Purpose: expose a browser UI where two AI agents (via ElevenLabs) chat about booking a hotel and switch into a data-over-sound protocol (ggwave) when they detect another AI. The demo uses WebAssembly (ggwave.wasm), custom API routes, and front-end components to orchestrate this.
File structure
• Top-level “hackathon_demo” folder contains:
• next.config.ts, tsconfig.json, tailwind.config.ts, package.json/lock → build & tooling
• public/ggwave → static JS/WASM for audio codec
• src/app → React entrypoints, layouts, global styles, API routes under /api/chat and /api/signed-url
• src/components → UI building blocks (AudioMessenger.tsx, ConvAI.tsx, Code.tsx, etc.)
• src/services → wrappers for chat & audio logic (AudioMessengerService.ts)
• src/utils & src/lib → shared helper functions and types
README
• Describes demo flow: two independent ElevenLabs AI agents converse, switch to ggwave for AI-to-AI.
• Mentions ggwave library (data-over-sound), ElevenLabs conversational API.
a. Main Components
• Frontend (Browser clients)
– React UI (Next.js pages & components)
– Audio capture/playback & ggwave codec (ggwave.js + ggwave.wasm)
• Next.js Server (API layer)
– /api/chat route: orchestrates message exchange between AI agents (text ↔ ElevenLabs)
– /api/signed-url route: provides pre-signed URLs (e.g., S3) for uploading/serving encoded audio blobs
– AudioMessengerService: server-side logic to manage audio message state
• External Services
– ElevenLabs Conversational AI & TTS API
– (Optional) Cloud storage for audio blobs (S3 or similar)
b. Relationships & Interactions
1. User’s browser captures audio → passes raw PCM to ggwave codec → produces encoded data blob.
2. Encoded blob uploaded via signed URL → stored in cloud storage.
3. Browser calls /api/chat with text or audio reference → Next.js server forwards to ElevenLabs → receives text response & TTS audio.
4. Server returns text + audio URL to client → browser fetches audio file or raw blob → ggwave decodes and plays.
5. Cycle repeats for each chat turn.
c. Architectural Patterns & Principles
• Jamstack style: separation of UI (static + dynamic React) & serverless functions (Next.js API routes).
• Componentized UI (Atomic design in src/components/ui).
• Service layer (AudioMessengerService) to abstract external API calls.
• Use of WebAssembly for performance-critical audio codec.
d. Core Technologies
• Next.js (React, App Router)
• Tailwind CSS for styling
• WebAssembly (ggwave.wasm + JS shim)
• ElevenLabs API (HTTP)
• Cloud storage (signed URLs, e.g., AWS S3)
• TypeScript, ts-node
Detail
• Every major component: React layout, API modules, service layer, static assets.
• Don’t omit the ggwave WASM module.
• Both text & audio flows separately or in parallel lanes for clarity.