How mralihasan/sophia-ai-assistant works
Project Type and Purpose
• This is a full-stack desktop AI assistant application.
• Purpose: Provide conversational AI features (via Hugging Face API), voice/text activation, OS automation (apps, websites, calls), and multimedia control, all wrapped in a local Python server + HTML/JS frontend.
File Structure
• Top‐level:
• engine/ — core Python logic (command parsing, configuration, DB access, feature implementations)
• main.py — launches the Python server (Eel or Bottle/Flask) and initializes the engine
• sophia.db — SQLite database for storing configuration, user data, custom apps/websites
• www/ — static frontend (HTML, CSS, JS, audio assets, vendor libraries)
• Key config files: engine/config.py
• Build/deploy: the Python runtime bundles the www folder via Eel (or Flask + bottle-websocket) to render a Chromium window as the UI.
Additional Insights from README
• Backend: Python + Eel (for desktop GUI) and/or Flask + bottle-websocket to serve API endpoints.
• Frontend: HTML/CSS/JS (uses Bootstrap, vendor scripts for text animation).
• Database: SQLite.
• External services: Hugging Face API for chat, OS APIs for speech recognition (pyaudio, pocketsphinx), automation (PyAutoGUI, PyGetWindow), telephony (pywhatkit).
Main Components
a. Frontend UI
• Browser window (Eel) hosting www/index.html, controller.js, main.js
• Assets: CSS (www/style.css), audio cues, vendor animations
b. Backend Server
• main.py → initializes Eel/Flask server
• HTTP/WebSocket endpoints (bottle-websocket or eel.expose) for user commands
• Speech recognition and TTS modules (PyAudio, pvporcupine, pyttsx3)
c. Engine Modules (engine/)
• command.py: maps textual commands to feature calls
• features.py: implements actions (open app, search YouTube, make calls, etc.)
• db.py: CRUD interface to sophia.db
• config.py: loads/stores configuration
d. Database
• SQLite file (sophia.db) storing user preferences, custom apps/websites
e. External Services
• Hugging Face API
• Operating System (Windows) for app control
• Web browsers (for site navigation / YouTube)
• Telephony/WhatsApp via pywhatkit
Relationships and Data Flow
• User speaks or types → Frontend captures input → Sent via WebSocket/HTTP to Backend
• Backend → SpeechRecognition (if voice) → engine.command.parse() → engine.features.execute()
• If NLP: Backend → Hugging Face API → engine.command receives response
• engine.features may read/write sophia.db via engine.db
• Result (text reply, OS action) returned → Backend sends it back to Frontend → UI updates + TTS speaks
Architectural Patterns & Design Principles
• Separation of concerns:
• Presentation layer (www/)
• Application layer (Python server & engine)
• Data layer (SQLite)
• Modular design: engine components decoupled (config, db, features)
• Event-driven communication: WebSocket or Eel RPC for real-time interactions
• Extensibility: add new features by extending engine/features.py and wiring commands in command.py