How encoredev/ai-chat works
Main Components
• Chat Service: Acts as the orchestrator that handles incoming messages from different platforms (Discord, Slack, Local).
• Provider Integrations:
– Chat Providers: Discord Service (under chat/provider/discord), Slack Service (under chat/provider/slack), and Local Chat Service (under chat/provider/local). Their directories include service files, database migration and query logic.
– LLM Providers: OpenAI and Gemini integrations are found in the llm/provider directory with corresponding client implementations.
• Bot Service: Handles the creation, storage, and management of bot profiles.
• LLM Service: Manages the crafting of prompts (including persona details), sending them to the LLM, processing streaming responses via pub/sub, and then communicating the results back to the Chat Service.
• Database Layers: Several services contain SQL migrations and query files (e.g., in bot/db, chat/service/db, chat/provider/discord/db) which represent persistent storage components.
• Infrastructure Services: Ancillary components include the pub/sub architecture for messaging between services as well as potential proxy handling (in the proxy directory) and commonly used utility functions (in pkg/fns).
Relationships and Interactions
• Data Flow from Users: Users interact with one of the chat interfaces (local web UI, Discord, or Slack). Their messages get received by the respective chat provider service.
• Messaging Pipeline: Each chat provider service publishes the message to the Chat Service (possibly using Encore’s pub/sub mechanism).
• Bot Interaction: Chat Service retrieves bot profiles and message histories from the Bot Service’s database layer and forwards the conversation context to the LLM Service.
• LLM Integration: The LLM Service uses provider-specific clients (OpenAI or Gemini) to send crafted prompts to the appropriate LLM, and responses are streamed back via pub/sub messaging.
• Response Handling: The LLM Service processes these responses and passes them back to the Chat Service, which then routes them to the correct chat platform.
Highlighted Architectural Patterns and Design Principles
• Microservices: The overall architecture is decomposed into service boundaries for each responsibility (chat orchestration, bot management, LLM prompt handling).
• Separation of Concerns: Each service handles a specific aspect – e.g., chat integrations are covered in a dedicated “chat” module while LLM integrations live within “llm”.
• Pub/Sub Communication: The design leverages pub/sub mechanisms to efficiently route messages between services.
• Environment-Agnostic Deployment: The use of Encore for streamlined local and cloud deployment, along with secrets management for API keys, demonstrates a design mindful of secure, scalable deployment.
• Provider Abstraction: The existence of separate provider directories indicates a flexible design that makes it easy to add or remove chat or LLM providers.
Including Relevant Technologies, Frameworks, or Libraries
• Encore (for rapid deployment and service orchestration)
• SQL scripts and migrations (for database initialization)
• Language-specific frameworks and libraries in Go (as seen from the go.mod file)
• NPM-based tooling, Tailwind, and Vite in the local chat front-end (for building the web UI)
• Pub/Sub messaging to integrate asynchronous communications across microservices