How alvinlee518/llmchat works
Overall Project Context and Purpose
• Project Type: LLMChat is an enterprise solution built in the Java ecosystem that combines large language model (LLM) functionality with LangChain integration. It supports AI chatbot applications, intelligent knowledge bases, and smart customer service.
• Purpose: To provide a robust, modular platform where companies can manage models (both local and external), handle knowledge management (e.g., segmentation, retrieval, embedding), and allow customizable AI applications—all backed by strong RBAC security.
Identifying the Main Components
a. Frontend
– Represent the “llmchat-ui” module which is built with Vue3, Vite, and TypeScript.
– Show its role as the client interface (web portal) that renders pages like model management, dataset views, and application debugging.
b. Backend – API Gateway and Business Layers
– Split the backend into several logical “containers” or modules:
• API Layer: The “llmchat-server-api” module exposing REST endpoints (label it “API Gateway”).
• Application Service Layer: The “llmchat-server-service” which coordinates business logic (services for AI Chat, Model, Document, etc.).
• Data Access Layer: The “llmchat-server-repository” handling persistence operations, MyBatis mappers, and entity models.
• Bootstrap/Startup Module: “llmchat-server-start” which acts as the launch point (Main application and controller dispatch).
c. Common Libraries (Shared Modules)
– “llmchat-common” subdivides into several functional libraries:
• Common Core (basic utilities, auto-configurations, exception handling).
• LangChain Integration (“llmchat-common-langchain”) for managing LLM providers, document parsers, retrievers, and other LangChain functionalities.
• MyBatis Integration (“llmchat-common-mybatis”) for data object mapping and code generation.
• Redis Integration (“llmchat-common-redis”) for caching and implementing message queues (e.g., via Redis Streams).
• Security (“llmchat-common-security”) for implementing RBAC, token management, and request handling using Spring Security.
d. External Systems and Dependencies
– Database: PostgreSQL (indicated in the README and likely used by MyBatis).
– Redis: For caching and pub/sub/streaming.
– Vector/Content Store: External services like Weaviate (for document and embedding search) and possibly MinIO for object storage.
– Other External APIs: Integration with external LLM providers (e.g., OpenAI, Azure OpenAI, etc.) and local model providers (LocalAI, Ollama, Xinference).
Architectural Patterns & Design Principles to Highlight
• Modular Monolithic/Layered Architecture: Emphasize how the project is split by Maven modules into API, service, repository, and common libraries.
• Separation of Concerns: The division between the frontend (presentation), gateway, business logic, data access, and integration packages.
• Extensibility: The “Common Langchain” module, in particular, is designed to support multiple LLM providers and document processing strategies.
• Asynchronous Messaging: Usage of Redis streams for handling asynchronous tasks (e.g., for processing document segmentation, embedding tasks).
• Security: Integration of RBAC and dedicated security modules using Spring Security.