How leon-ai/leon works
Project Type and Core Purpose
• Leon is an open‑source personal assistant that allows voice and text interactions.
• Its purpose is to provide a modular system for AI personal assistance, including skills (plugins), various communication bridges (Node.js and Python), and integrations with voice engines, hotword detection, and a web interface.
• The system is designed to be extendable (skills directory) and supports offline operation, privacy, and multiple channels (HTTP, TCP).
Main Components (nodes) from the File Structure
a. Web Frontend (app):
– Located under the “app” directory with HTML, CSS (SCSS), JavaScript, and React (JSX) components.
– It provides the user interface for interacting with Leon.
b. Core Server and API (server):
– The “server” directory contains the core backend logic, including HTTP server endpoints, NLP modules, and components managing ASR, TTS, conversation flow, and an LLM manager.
– It also includes middleware (plugins, logging, utilities) and various API endpoints (e.g., fetch-widget, run-action).
c. Bridges:
– Two separate bridges (Node.js and Python) found in the “bridges” directory. They handle the integration between the core system and external or language-specific modules.
– Their SDK directories expose functionalities (Aurora components, settings, etc.) to skills and core components.
d. Skills:
– The “skills” directory holds many individual skill modules (each with its own configuration, actions, library, and tests).
– Skills are the modular, pluggable components that define Leon’s capabilities (for games, finance, productivity, etc.).
e. Additional Nodes:
– The “hotword” node, which includes voice processing models for hotword detection.
– The “tcp_server” that appears to manage inter-process communication (for connecting with third-party nodes or modules like spaCy).
– The “core” directory holds shared configurations, data models, languages, and global settings used across Leon.
– The “scripts” directory – containing build, setup, release, and training scripts – indicates that development and deployment are streamlined with automation.
Relationships and Data Flow
• Users interact with Leon over a web frontend (in “app”) or via text/voice.
• The frontend communicates with the backend server via HTTP APIs.
• The core server acts as the brain of the assistant: it manages conversations, interacts with language models (LLM, NLP) and calls appropriate skills.
• Skills are independently structured modules that plug into the core (through standardized config, memory, and action interfaces).
• The bridges (Node.js and Python) are responsible for communication between the core server and other language-specific modules, allowing for a hybrid approach (supporting both JavaScript/TypeScript and Python-based skills or processing).
• The hotword component continuously listens for activation phrases and hands off control to the core server.
• Data flows among these components are orchestrated via API endpoints on the HTTP server, TCP channels (tcp_server), and SDK calls provided by the bridges.
Architectural Patterns and Design Principles
• Modular/Micro-kernel Architecture: The core system remains lean while skills extend functionality. Each skill is self-contained.
• Separation of Concerns: Clear partitioning is seen between user interface (frontend), core processing (server/core), integrations (bridges, TCP server, hotword), and skills.
• Use of SDKs: The bridges expose well-defined APIs (Aurora SDK-libraries) indicating a clear abstraction layer for external modules.
• Configuration-Driven: Multiple JSON config files (in various directories) indicate that many components are highly configurable and decoupled from code.