How culyrh/agentic-kiosk works
This is a FastAPI voice-ordering kiosk backend: a customer streams speech, the server detects an utterance, transcribes it, and lets a LangChain agent choose menu and cart tools before returning streamed speech and kiosk actions. The product also exposes REST menu, cart, order, option, set, and semantic-search APIs. Its key internal layers are session-aware tools, hybrid menu retrieval, and SQLite persistence. Edges reflect the README or shown calls/import references; several detailed capabilities are documented but their implementation was not sampled.
Main entry points: main.py FastAPI app · sqlite.py SQLite access · stt.py Voice WebSocket · agent.py ReAct agent
Components 21
Voice interaction
- Voice WebSocket
api/routes/stt.py - Speech detection
voice/vad_silero.py - Speech recognition
voice/stt.py - Speech synthesis
voice/tts.py
Agent and retrieval
- ReAct agent
app/agent.py - Session context
app/session_context.py - Latency tracking
app/latency_tracker.py - Menu tools
app/tools/menu_tools.py - Cart and payment tools
app/tools/cart_tools.py - Semantic retrieval
app/rag/search.py
Ordering API
- FastAPI app
api/main.py - Menu API
api/routes/menu.py - Cart API
api/routes/cart.py - Order API
api/routes/order.py - Search API
api/routes/search.py - Options API
api/routes/options.py - Sets API
api/routes/sets.py
Data and persistence
- Vector index
app/rag/chroma.py - SQLite access
db/sqlite.py - Menu and orders DB
data/ria_menu.db
Other components
- Kiosk customer
Connections 28
- Kiosk customer → FastAPI app: sends requests
README.md - FastAPI app → Voice WebSocket: routes to
api/main.py - FastAPI app → Menu API: routes to
api/main.py - FastAPI app → Cart API: routes to
api/main.py - FastAPI app → Order API: routes to
api/main.py - FastAPI app → Search API: routes to
api/main.py - FastAPI app → Options API: routes to
api/main.py - FastAPI app → Sets API: routes to
api/main.py - Voice WebSocket → Speech detection: detects speech
api/routes/stt.py - Voice WebSocket → Speech recognition: transcribes utterance
api/routes/stt.py - Voice WebSocket → ReAct agent: handles transcript
api/routes/stt.py - Voice WebSocket → Speech synthesis: synthesizes response
api/routes/stt.py - ReAct agent → Menu tools: dispatches tools
app/agent.py - ReAct agent → Cart and payment tools: dispatches tools
app/agent.py - ReAct agent → Session context: tracks conversation
app/agent.py - ReAct agent → Latency tracking: records timings
app/agent.py - Menu tools → Vector index: searches vectors
app/tools/menu_tools.py - Menu tools → Menu and orders DB: queries menu data
app/tools/menu_tools.py - Cart and payment tools → Session context: reads session ID
app/tools/cart_tools.py - Cart and payment tools → Menu and orders DB: updates cart and orders
app/tools/cart_tools.py - Search API → Semantic retrieval: calls search
api/routes/search.py - Semantic retrieval → Vector index: runs similarity search
app/rag/search.py - Menu API → SQLite access: queries menus
api/routes/menu.py - Cart API → SQLite access: reads and updates cart
api/routes/cart.py - Order API → SQLite access: queries orders
api/routes/order.py - Options API → SQLite access: queries options
api/routes/options.py - Sets API → SQLite access: queries sets
api/routes/sets.py - SQLite access → Menu and orders DB: reads and writes
db/sqlite.py