# easeaico/llm_gateway architecture

> Architecture diagram of the GitHub repository easeaico/llm_gateway, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/easeaico/llm_gateway
- Repository: https://github.com/easeaico/llm_gateway

## Overview

##### Project Type and Purpose
- The project is a specialized system that integrates multiple large language models (LLMs) into a unified mesh.
- It is designed to provide an interface (such as chat functionality) that connects to various LLM providers, abstracting their differences.

##### File Structure
- Top-Level Directories/Files: You have core Go files (main.go, chat.go, clients.go, config.go) along with configuration (config.yaml.example) and deployment (Dockerfile, Makefile) files.
- This indicates a monolithic service written in Go that encapsulates its configuration, client adapters for each LLM, and chat or conversation logic.

##### README for Additional Insights
- The brief README explains that this is a “mesh system for adapting multiple large language models,” which confirms that the application serves as an intermediary layer to communicate with various LLM backends.

##### Main Components
a. Entry Point and Configuration:
- Main.go as the system’s entry point.
- Configuration component (config.go, along with config.yaml.example) to handle dynamic settings, such as endpoint URLs and API keys for different LLMs.
b. Chat Handling:
- Use “chat.go” to illustrate the chat engine or conversation management module that handles incoming chat queries and orchestrates responses.
c. Client Adapters:
d. External LLM Services:
e. Deployment/Runtime Environment:
- Docker container (from Dockerfile) to show how the application is packaged for deployment.
- Optionally, illustrate the Makefile/build process as part of the continuous integration/deployment pipeline.

##### Relationships and Interactions
- Directional flow starting from incoming requests (e.g., a client or user) reaching main.go.
- main.go consults the configuration module to load parameters.
- It then routes the request to the chat module (chat.go), which processes the communication.
- The chat module interacts with the clients component (clients.go) to dispatch requests to one or more external LLM services.
- Responses flow back through the same chain to reach the end-user.

##### Architectural Patterns and Design Principles
- Separation of concerns: distinct files for configuration, chat logic, and client adapters demonstrate modular design.
- Extensibility: The design allows for adding new client adapters to integrate additional LLM services without significant changes to the core logic.
- Containerization: Inclusion of Dockerfile indicates that the system is containerized for consistent deployment.

## Diagram (Mermaid)

```mermaid
graph TD
    %% External User
    User["User/Client"]:::user

    %% Go Application Modules
    subgraph "Go Application Modules"
        mainEntry["Main Entry (main.go)"]:::internal
        configModule["Configuration Loader (config.go)"]:::internal
        configYAML["Config YAML (config.yaml.example)"]:::internal
        chatEngine["Chat Engine (chat.go)"]:::internal
        clientAdapters["LLM Client Adapters (clients.go)"]:::internal
    end

    %% Deployment Environment
    subgraph "Deployment"
        dockerEnv["Docker Container (Dockerfile)"]:::deployment
        makeEnv["Build/CI (Makefile)"]:::deployment
    end

    %% External LLM Services
    subgraph "External LLM Services"
        LLM1["LLM Service 1"]:::external
        LLM2["LLM Service 2"]:::external
    end

    %% Flow Connections
    User -->|"sendsRequest"| mainEntry
    mainEntry -->|"loadsConfig"| configModule
    configModule -->|"reads"| configYAML
    mainEntry -->|"routesToChat"| chatEngine
    chatEngine -->|"dispatchesReq"| clientAdapters
    clientAdapters -->|"callsLLM"| LLM1
    clientAdapters -->|"callsLLM"| LLM2
    LLM1 -->|"sendsResponse"| clientAdapters
    LLM2 -->|"sendsResponse"| clientAdapters
    clientAdapters -->|"aggregatesResponse"| chatEngine
    chatEngine -->|"returnsResponse"| mainEntry
    mainEntry -->|"sendsResponse"| User

    %% Deployment Links
    mainEntry -.->|"deployedVia"| dockerEnv
    makeEnv -.->|"buildsFor"| dockerEnv

    %% Click Events
    click mainEntry "https://github.com/easeaico/llm_gateway/blob/main/main.go"
    click configModule "https://github.com/easeaico/llm_gateway/blob/main/config.go"
    click configYAML "https://github.com/easeaico/llm_gateway/blob/main/config.yaml.example"
    click chatEngine "https://github.com/easeaico/llm_gateway/blob/main/chat.go"
    click clientAdapters "https://github.com/easeaico/llm_gateway/blob/main/clients.go"
    click dockerEnv "https://github.com/easeaico/llm_gateway/tree/main/Dockerfile"
    click makeEnv "https://github.com/easeaico/llm_gateway/tree/main/Makefile"

    %% Styles
    classDef user fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef internal fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef deployment fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
```
