How deluxer/oliva works
Main Components
• Voice Assistant Module:
• Located under app/voice_assistant, this component handles real-time voice input.
• Integrates with Livekit for real-time communication and with a speech-to-text service (e.g., Deepgram) for voice conversion.
• Agent Components:
• Under app/agents, these include:
▪ Core: (app/agents/core) Base classes handling state management and common interfaces.
▪ Implementations: (app/agents/implementations) Specific agents – for example, blog post processing, Amazon product searches (both JSON and superlinked versions).
▪ Supervisor: A coordinator (found in app/agents/implementations/supervisor.py) that routes voice commands and queries to the appropriate agent.
• LangChain Integration:
• Located in app/agents/langchain, it contains several sub-components:
▪ Nodes (app/agents/langchain/nodes): Represent the discrete processing steps (e.g., agent logic, generate, rewrite).
▪ Edges (app/agents/langchain/edges): Manage conditional transitions between nodes.
▪ Tools (app/agents/langchain/tools): Provide specific functionality like Amazon product search and blog post tools.
▪ Vector Store (app/agents/langchain/vector_store): Implements document retrieval functionality interfacing with the Qdrant vector database.
▪ Factory and Interface: Used for initializing and managing LangChain components.
• Utilities:
• In app/utils, common helper functions and constants are shared across modules.
Relationships and Interactions
• The Voice Assistant receives user input via Livekit, converts speech to text, and forwards the request.
• The Supervisor agent determines which specific agent should be triggered next based on the input (implementing the core multi-agent decision logic).
• Each agent implementation processes the request using:
• Interactions with external services – for example, fetching vectors from Qdrant or running semantic searches with Superlinked.
• Data flows from the user (voice input) to the voice assistant module, then to the supervisor agent, and finally through the appropriate agent’s workflow (utilizing LangChain’s nodes/edges) before potentially returning a result.
Architectural Patterns and Design Principles
• Modular Architecture: Clear separation between voice input, supervision, agent-specific implementations, and shared utilities.
• Integration of External Services: Clearly delineated responsibilities for external services (Livekit for real-time, Qdrant for vector storage, Superlinked for semantic search, Deepgram as speech-to-text).
• Extensibility: Each agent implementation (placed under app/agents/implementations) can be enhanced or replaced independently while maintaining consistent communication protocols through core interfaces.
Relevant Technologies
• LangChain (LLM integration, nodes, edges, tools, and vector store)
• Livekit (real-time voice communication)
• Qdrant (vector database for semantic search)
• Superlinked (framework for semantic search)
• Deepgram (speech-to-text conversion)
• Python (version 3.12+ as the core implementation language)
• Detail dynamic routing: Show how the supervisor agent directs requests to different agent implementations.