# xynehq/xyne architecture

> Architecture diagram of the GitHub repository xynehq/xyne, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xynehq/xyne
- Repository: https://github.com/xynehq/xyne

## Overview

##### Project Type and Purpose
- Xyne is an AI-first search and answer engine designed for the enterprise workspace. It is intended to securely ingest data from various SaaS applications (Google Workspace, Slack, Atlassian, etc.), build a relationship graph and then provide quick, contextual answers.
- It is a full-stack solution with both a web frontend and a robust backend, and it emphasizes secure, permission-aware data handling with integrations to external services.

##### High-Level Architectural Components
a. Frontend:
- Located under the “frontend” directory. This part is built with modern JavaScript/TypeScript (using Vite, Tailwind CSS, etc.) and handles the user interface and interactions such as search, chat, and OAuth flows.
- Key files: frontend/src/main.tsx, components, routes and configuration files.

b. Backend (Server):
- Found in the “server” directory, it encapsulates multiple concerns:
- API endpoints (e.g., admin, auth, chat, oauth, proxy, search) which serve as the interconnection point for the frontend.
- AI integrations under server/ai and multiple providers – these modules integrate with various LLMs/models (OpenAI, Bedrock, Gemini, etc.) and likely preprocess or format user queries.
- Database handling in server/db – showing model definitions for chat, user, oauth providers, etc.
- Evaluation and search modules (server/eval, server/search, server/vespa) indicate that there are specific pipelines to evaluate queries and perform search indexing/retrieval.
- The “integrations” directory shows connections to external services such as Google services, which is central given the Google Workspace integration described in the README.
- Utility modules (logger, errors, queue, shared, etc.) support the overall operations.

c. Deployment and Infrastructure:
- Multiple deployment configurations are provided through Dockerfiles, docker-compose files (for dev, prod, self-hosted, metrics) and even scripts for cloud (e.g., AWS) setups in both deployment/ and start-aws-linux.sh.
- There are configuration and environment files (.env.template, etc.) to ensure proper deployment across various infrastructures.

d. Observability:
- Under “observability”, there are docs and setup configurations (Grafana metrics, Prometheus configurations) which underline the importance of monitoring and performance tracking.

##### Interactions and Relationships
- The frontend communicates with the backend server through REST APIs (or similar endpoints) across the different modules such as auth, chat, search, etc.
- The backend handles business logic by connecting to:
- The AI providers for processing queries.
- The database layer for storing and retrieving persistent data (users, chat, workspace, etc.).
- External integration services (Google, etc.) to ingest and synchronize data from various SaaS apps.
- The evaluation pipeline (found in the server/eval directory) processes query performance and data integrity, then passes results possibly to the search module (server/search) to fetch results from the deployed Vespa search clusters.
- A messaging/queue system (server/queue) orchestrates asynchronous processing tasks, e.g., for ingestion or long-running AI tasks.
- Deployment and orchestration are done using containerized setups (Docker, docker-compose), encapsulating each component into its deployment module. Observability modules (e.g., Prometheus, Grafana) monitor services and metrics.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Clear division between the UI (frontend) and the backend (server) with APIs governing the communication.
- Modular Design: The backend is split into distinct directories (ai, api, db, eval, integrations) each managing a different aspect of the system.
- Extensibility and Integration: The modular integrations for external data sources and AI providers suggest an architecture designed for plugging in various external services without disrupting core flows.
- Containerization & Cloud Readiness: The presence of multiple Dockerfiles and deployment manifests shows containered deployments and support for cloud environments.
- Observability: Deployment of monitoring services ensures that the system’s health and performance are continuously tracked.

- Use clear labels for each component mentioned above.
- From the frontend UI sending requests to the backend API.
- Between backend modules (for example, API endpoints communicating with the database or forwarding queries to AI integrators).
- From external services (Google Workspace, Slack, etc.) flowing data into the integrations layer.
- For example, rectangles for service components (frontend, API, AI Providers).
- Areas that are modular (the extensible integration points and provider modules) so that it’s clear where new functionality can be connected.
- Detail the internal AI and search pipeline, showing how a user query transitions from the frontend through the AI evaluation modules to the final search responses.
- Lastly, annotate critical components with notes on security and performance (e.g., “Permissions-aware”, “Multi-threaded ingestion”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Actor
    User("User"):::external

    %% Frontend Subgraph
    subgraph "Frontend"
        FrontendUI("Frontend UI (Vite, Tailwind)"):::frontend
    end

    %% Backend Subgraph
    subgraph "Backend"
        API("API Endpoints (auth,chat,oauth,search)"):::backend
        AI("AI Providers (OpenAI,Bedrock,Gemini)"):::backend
        DB("Database (chat,user,oauthProvider,workspace)"):::backend
        Eval("Evaluation Module (evaluate,processDataForVespa)"):::backend
        Search("Search Module (vespa,vespaClient,mappers)"):::backend
        Ext("External Integrations (Google services,etc)"):::backend
        Queue("Messaging/Queue System (async tasks)"):::backend
    end

    %% Deployment and Observability Nodes
    Deploy("Deployment & Infrastructure (Docker,docker-compose,cloud scripts)"):::infra
    Observ("Observability (Grafana,Prometheus)"):::observability

    %% Relationships
    User -->|"initiates"| FrontendUI
    FrontendUI -->|"sends requests"| API

    API -->|"calls"| DB
    API -->|"calls"| AI
    API -->|"calls"| Ext
    API -->|"triggers"| Queue
    API -->|"triggers"| Eval
    AI -->|"provides-data"| Eval
    Eval -->|"processes-query"| Search

    %% Deployment relationships
    FrontendUI -->|"deployed on"| Deploy
    API -->|"deployed on"| Deploy

    %% Observability relationships (dashed lines)
    Deploy -.->|"monitoredBy"| Observ
    FrontendUI -.->|"monitoredBy"| Observ
    API -.->|"monitoredBy"| Observ

    %% Click Events
    click FrontendUI "https://github.com/xynehq/xyne/tree/main/frontend/"
    click API "https://github.com/xynehq/xyne/tree/main/server/api"
    click AI "https://github.com/xynehq/xyne/tree/main/server/ai"
    click DB "https://github.com/xynehq/xyne/tree/main/server/db"
    click Eval "https://github.com/xynehq/xyne/tree/main/server/eval"
    click Search "https://github.com/xynehq/xyne/tree/main/server/search"
    click Ext "https://github.com/xynehq/xyne/tree/main/server/integrations/google"
    click Queue "https://github.com/xynehq/xyne/tree/main/server/queue"
    click Deploy "https://github.com/xynehq/xyne/tree/main/deployment/"
    click Observ "https://github.com/xynehq/xyne/tree/main/observability/"

    %% Styles
    classDef frontend fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef backend fill:#c8e6c9,stroke:#1b5e20,stroke-width:2px;
    classDef infra fill:#d7ccc8,stroke:#4e342e,stroke-width:2px;
    classDef observability fill:#ffccbc,stroke:#bf360c,stroke-width:2px;
    classDef external fill:#e0e0e0,stroke:#757575,stroke-width:2px, font-style:italic;
```
