# dataease/sqlbot architecture

> Architecture diagram of the GitHub repository dataease/sqlbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dataease/sqlbot
- Repository: https://github.com/dataease/sqlbot

## Overview

1. **What this repository is**
   - SQLBot is a full-stack “chat with data” system: users ask questions in natural language, the backend turns them into SQL and analysis steps with LLM/RAG support, and the frontend renders answers as charts, tables, or narrative output.
   - The codebase is organized as a Python backend, a Vue 3 frontend, and a small Node-based chart SSR service.

2. **Top-level runtime layout**
   - The main application runs as a backend API service plus a single-page frontend.
   - Docker is the primary packaging/deployment path, with separate Dockerfiles and a compose-based installer flow.
   - Persistent state is split across a relational database, file volumes for uploads/logs/images, and external model/data integrations.

3. **Backend architecture**
   - `backend/main.py` and `backend/apps/api.py` form the entry points into a modular FastAPI-style backend.
   - Feature code is organized by domain under `backend/apps/*`, while shared infrastructure lives in `backend/common/*`.
   - Database schema is managed with Alembic migrations in `backend/alembic/versions`, indicating a strongly versioned relational model.

4. **Core infrastructure boundaries**
   - `backend/apps/db` centralizes database and engine abstractions, including a separate `es_engine.py`, which suggests the backend can route some retrieval/search workloads through a search engine layer.
   - `backend/common/core` contains cross-cutting concerns: config, dependency wiring, pagination, file helpers, security, caching, and response middleware.
   - This separation keeps business modules thin and lets the app share auth, locale, and response behavior consistently.

5. **Identity, tenant, and platform management**
   - The `system` module owns users, login, workspaces, assistants, API keys, authentication providers, platform parameters, and system variables.
   - Workspace is a key boundary in the architecture: it is used to isolate resources and permissions rather than treating the app as a single global tenant space.
   - Authentication also supports multiple enterprise login modes, reflected by the xpack-specific frontend login views and the backend auth middleware.

6. **Datasource and governance domain**
   - `backend/apps/datasource` is one of the main domains: it manages data sources, tables, fields, permissions, row permissions, table relations, and “recommended problem” prompts.
   - This module also includes embedding-related code for datasource/table embeddings, which is part of the RAG layer used to improve text-to-SQL matching.
   - Excel utilities and datasource-specific helpers show that structured file imports are part of the data onboarding path.

7. **AI model abstraction**
   - `backend/apps/ai_model` abstracts LLM and embedding providers behind a factory layer.
   - The tree shows a native OpenAI implementation plus OpenAI-compatible provider support, matching the README’s list of supported model vendors.
   - This provider abstraction is the main boundary between SQLBot’s prompt/agent logic and the external model APIs.

8. **Chat and question-answer execution flow**
   - `backend/apps/chat` handles the end-user question lifecycle: chat APIs, CRUD/state management, chat models, and task execution.
   - `backend/apps/chat/task/llm.py` is where the backend likely orchestrates the model call, SQL generation, and answer assembly.
   - The template subsystem under `backend/apps/template` is central to this flow: it contains generators for SQL, charts, analysis, predictions, dynamic prompts, guess-question flows, datasource selection, and filtering.

9. **Knowledge refinement and prompt assets**
   - `backend/apps/terminology` stores domain terminology used to improve query interpretation and SQL generation.
   - `backend/apps/data_training` appears to manage training/feedback data used to improve future answers.
   - The backend also stores reusable SQL examples and template YAML assets under `backend/templates`, which are likely used to calibrate prompt behavior across different databases.

10. **Dashboard and visualization subsystem**
   - `backend/apps/dashboard` manages dashboard metadata and CRUD logic, while the frontend contains a substantial canvas/editor/preview stack for composing dashboard layouts.
   - The frontend chart components and the `g2-ssr` service indicate chart rendering is split between client-side editing and server-side chart generation/rendering support.
   - This makes dashboards a separate product surface from chat: one is conversational analytics, the other is persisted visual composition.

11. **Audit, logs, and operational traceability**
   - `backend/common/audit` provides request-context and logging decorators, and the migrations include system logs and chat logs tables.
   - This suggests the system records not only login/admin activity but also the execution trace of chat interactions and generated outputs.
   - Those logs are important for debugging, traceability, and improving generated answers over time.

12. **Frontend structure**
   - The frontend is a Vue 3 + Vite SPA with domain-based structure: `api/`, `stores/`, `router/`, `components/`, and `views/`.
   - Major view groups mirror backend domains: chat, dashboard, datasource management, system administration, embedded mode, login, and workspaces.
   - Internationalization is built in with multiple locale files, and there are dedicated embedded/UI helper assets for assistant and swagger-like documentation experiences.

13. **Integration and embedding**
   - SQLBot is designed to be embedded into other products, reflected by `frontend/src/views/embedded`, `frontend/public/assistant.js`, and the README’s emphasis on web embedding, popup embedding, and MCP usage.
   - The `backend/apps/mcp` module is the backend integration boundary for external agent/tool calling.
   - In architecture terms, SQLBot is not just a standalone app; it is also an embeddable AI data-answering service.

14. **Repository-level delivery and operations**
   - CI/CD workflows build and push container images, run checks, and sync to alternate registries.
   - Installer scripts and compose files support packaged deployment in constrained environments, including offline-style installation flows.
   - Overall, the repo is structured for enterprise self-hosting: versioned schema, containerized runtime, workspace isolation, and configurable external LLM backends.

Main entry points: [`backend/apps/api.py`](https://github.com/dataease/sqlbot/blob/main/backend/apps/api.py) (API router), [`frontend/src`](https://github.com/dataease/sqlbot/tree/main/frontend/src) (SPA shell), [`backend/apps/template`](https://github.com/dataease/sqlbot/tree/main/backend/apps/template) (Templates), [`installer`](https://github.com/dataease/sqlbot/tree/main/installer) (Deploy)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_backend["Backend"]
  node_backend_main(("App entry<br/>FastAPI entry<br/>[main.py]"))
  node_backend_api["API router<br/>HTTP assembly<br/>[api.py]"]
  node_backend_core["Core<br/>shared runtime"]
  node_backend_audit["Audit<br/>logging trace"]
  node_backend_db[("DB layer<br/>db abstractions")]
  node_backend_system["System<br/>identity/admin"]
  node_backend_auth["Auth middleware<br/>security filter<br/>[auth.py]"]
  node_backend_datasource["Datasource<br/>data governance"]
  node_backend_embeddings{{"Embeddings<br/>RAG indexing"}}
  node_backend_ai_model["AI models<br/>LLM factory"]
  node_backend_chat["Chat<br/>conversation flow"]
  node_backend_template["Templates<br/>prompt pipeline"]
  node_backend_dashboard["Dashboard"]
  node_backend_terminology["Terminology<br/>query hints"]
  node_backend_training["Training<br/>feedback data"]
  node_backend_mcp["MCP<br/>tool bridge"]
end

subgraph group_frontend["Frontend"]
  node_frontend_app(("SPA shell<br/>Vue app"))
  node_frontend_chat["Chat UI<br/>analytics UI"]
  node_frontend_dashboard["Dashboard UI<br/>canvas editor"]
  node_frontend_datasource["Datasource UI<br/>admin UI"]
  node_frontend_system["System UI<br/>admin UI"]
  node_frontend_embedded["Embedded UI<br/>embed surface"]
end

subgraph group_infra["Infra"]
  node_g2_ssr{{"Chart SSR<br/>node renderer"}}
  node_installer_deploy["Deploy<br/>packaging"]
end

node_backend_main -->|"starts"| node_backend_api
node_backend_api -->|"uses"| node_backend_core
node_backend_api -->|"logs"| node_backend_audit
node_backend_api -->|"persists"| node_backend_db
node_backend_api -->|"mounts"| node_backend_system
node_backend_api -->|"mounts"| node_backend_datasource
node_backend_api -->|"mounts"| node_backend_chat
node_backend_api -->|"mounts"| node_backend_dashboard
node_backend_api -->|"mounts"| node_backend_terminology
node_backend_api -->|"mounts"| node_backend_training
node_backend_api -->|"exposes"| node_backend_mcp
node_backend_system -->|"enforces"| node_backend_auth
node_backend_datasource -->|"indexes"| node_backend_embeddings
node_backend_chat -->|"calls"| node_backend_ai_model
node_backend_chat -->|"prompts"| node_backend_template
node_backend_template -->|"selects"| node_backend_datasource
node_backend_template -->|"uses"| node_backend_terminology
node_backend_template -->|"formats"| node_backend_ai_model
node_frontend_app -->|"renders"| node_frontend_chat
node_frontend_app -->|"renders"| node_frontend_dashboard
node_frontend_app -->|"renders"| node_frontend_datasource
node_frontend_app -->|"renders"| node_frontend_system
node_frontend_app -->|"renders"| node_frontend_embedded
node_frontend_chat -->|"uses"| node_g2_ssr
node_frontend_dashboard -->|"uses"| node_g2_ssr
node_installer_deploy -->|"packages"| node_backend_main
node_installer_deploy -->|"packages"| node_frontend_app
node_installer_deploy -->|"packages"| node_g2_ssr

click node_backend_main "https://github.com/dataease/sqlbot/blob/main/backend/main.py"
click node_backend_api "https://github.com/dataease/sqlbot/blob/main/backend/apps/api.py"
click node_backend_core "https://github.com/dataease/sqlbot/tree/main/backend/common/core"
click node_backend_audit "https://github.com/dataease/sqlbot/tree/main/backend/common/audit"
click node_backend_db "https://github.com/dataease/sqlbot/tree/main/backend/apps/db"
click node_backend_system "https://github.com/dataease/sqlbot/tree/main/backend/apps/system"
click node_backend_auth "https://github.com/dataease/sqlbot/blob/main/backend/apps/system/middleware/auth.py"
click node_backend_datasource "https://github.com/dataease/sqlbot/tree/main/backend/apps/datasource"
click node_backend_embeddings "https://github.com/dataease/sqlbot/tree/main/backend/apps/datasource/embedding"
click node_backend_ai_model "https://github.com/dataease/sqlbot/tree/main/backend/apps/ai_model"
click node_backend_chat "https://github.com/dataease/sqlbot/tree/main/backend/apps/chat"
click node_backend_template "https://github.com/dataease/sqlbot/tree/main/backend/apps/template"
click node_backend_dashboard "https://github.com/dataease/sqlbot/tree/main/backend/apps/dashboard"
click node_backend_terminology "https://github.com/dataease/sqlbot/tree/main/backend/apps/terminology"
click node_backend_training "https://github.com/dataease/sqlbot/tree/main/backend/apps/data_training"
click node_backend_mcp "https://github.com/dataease/sqlbot/tree/main/backend/apps/mcp"
click node_frontend_app "https://github.com/dataease/sqlbot/tree/main/frontend/src"
click node_frontend_chat "https://github.com/dataease/sqlbot/tree/main/frontend/src/views/chat"
click node_frontend_dashboard "https://github.com/dataease/sqlbot/tree/main/frontend/src/views/dashboard"
click node_frontend_datasource "https://github.com/dataease/sqlbot/tree/main/frontend/src/views/ds"
click node_frontend_system "https://github.com/dataease/sqlbot/tree/main/frontend/src/views/system"
click node_frontend_embedded "https://github.com/dataease/sqlbot/tree/main/frontend/src/views/embedded"
click node_g2_ssr "https://github.com/dataease/sqlbot/tree/main/g2-ssr"
click node_installer_deploy "https://github.com/dataease/sqlbot/tree/main/installer"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_backend_main,node_backend_api,node_backend_core,node_backend_audit,node_backend_db,node_backend_system,node_backend_auth,node_backend_datasource,node_backend_embeddings,node_backend_ai_model,node_backend_chat,node_backend_template,node_backend_dashboard,node_backend_terminology,node_backend_training,node_backend_mcp toneBlue
class node_frontend_app,node_frontend_chat,node_frontend_dashboard,node_frontend_datasource,node_frontend_system,node_frontend_embedded toneAmber
class node_g2_ssr,node_installer_deploy toneMint
```

## Components

### Backend

Python API service and domain modules

- [App entry](https://github.com/dataease/sqlbot/blob/main/backend/main.py): FastAPI entry · Bootstraps the backend service and mounts API modules · `backend/main.py`
- [API router](https://github.com/dataease/sqlbot/blob/main/backend/apps/api.py): HTTP assembly · Wires domain routers and middleware into the application · `backend/apps/api.py`
- [Core](https://github.com/dataease/sqlbot/tree/main/backend/common/core): shared runtime · Cross-cutting config, deps, security, pagination, files, caching, and responses · `backend/common/core`
- [Audit](https://github.com/dataease/sqlbot/tree/main/backend/common/audit): logging trace · Request context and log decoration for operational traceability · `backend/common/audit`
- [DB layer](https://github.com/dataease/sqlbot/tree/main/backend/apps/db): db abstractions · Database engine, SQL helpers, constants, and search engine integration · `backend/apps/db`
- [System](https://github.com/dataease/sqlbot/tree/main/backend/apps/system): identity/admin · Users, login, workspaces, assistants, API keys, parameters, and platform variables · `backend/apps/system`
- [Auth middleware](https://github.com/dataease/sqlbot/blob/main/backend/apps/system/middleware/auth.py): security filter · Authentication and enterprise login enforcement · `backend/apps/system/middleware/auth.py`
- [Datasource](https://github.com/dataease/sqlbot/tree/main/backend/apps/datasource): data governance · Data sources, tables, fields, permissions, row rules, relations, and recommended problems · `backend/apps/datasource`
- [Embeddings](https://github.com/dataease/sqlbot/tree/main/backend/apps/datasource/embedding): RAG indexing · Datasource and table embeddings used for retrieval and text-to-SQL matching · `backend/apps/datasource/embedding`
- [AI models](https://github.com/dataease/sqlbot/tree/main/backend/apps/ai_model): LLM factory · Provider abstraction for LLM and embedding backends · `backend/apps/ai_model`
- [Chat](https://github.com/dataease/sqlbot/tree/main/backend/apps/chat): conversation flow · Chat state, CRUD, task execution, and answer assembly · `backend/apps/chat`
- [Templates](https://github.com/dataease/sqlbot/tree/main/backend/apps/template): prompt pipeline · Generators for SQL, charts, analysis, predictions, dynamic prompts, datasource selection, and filters · `backend/apps/template`
- [Dashboard](https://github.com/dataease/sqlbot/tree/main/backend/apps/dashboard): dashboard domain · Persisted dashboard metadata and CRUD services · `backend/apps/dashboard`
- [Terminology](https://github.com/dataease/sqlbot/tree/main/backend/apps/terminology): query hints · Domain vocabulary used to improve interpretation and SQL generation · `backend/apps/terminology`
- [Training](https://github.com/dataease/sqlbot/tree/main/backend/apps/data_training): feedback data · Stores data used to refine future answers · `backend/apps/data_training`
- [MCP](https://github.com/dataease/sqlbot/tree/main/backend/apps/mcp): tool bridge · Integration boundary for external agent/tool calling · `backend/apps/mcp`

### Frontend

Vue SPA and UI feature surfaces

- [SPA shell](https://github.com/dataease/sqlbot/tree/main/frontend/src): Vue app · Vite/Vue application shell, router, stores, and shared UI runtime · `frontend/src`
- [Chat UI](https://github.com/dataease/sqlbot/tree/main/frontend/src/views/chat): analytics UI · Conversation views, answers, execution logs, and chart rendering components · `frontend/src/views/chat`
- [Dashboard UI](https://github.com/dataease/sqlbot/tree/main/frontend/src/views/dashboard): canvas editor · Dashboard editor, preview, canvas layout, and component library · `frontend/src/views/dashboard`
- [Datasource UI](https://github.com/dataease/sqlbot/tree/main/frontend/src/views/ds): admin UI · Datasource, tables, permissions, relations, and upload flows · `frontend/src/views/ds`
- [System UI](https://github.com/dataease/sqlbot/tree/main/frontend/src/views/system): admin UI · User, model, auth, workspace, prompt, training, variables, and platform administration · `frontend/src/views/system`
- [Embedded UI](https://github.com/dataease/sqlbot/tree/main/frontend/src/views/embedded): embed surface · Embeddable assistant and page views for integration into other products · `frontend/src/views/embedded`

### Infra

Deployment, SSR charting, and packaging

- [Chart SSR](https://github.com/dataease/sqlbot/tree/main/g2-ssr): node renderer · Server-side chart rendering service for exported or embedded visualizations · `g2-ssr`
- [Deploy](https://github.com/dataease/sqlbot/tree/main/installer): packaging · Docker, compose, installer scripts, and release workflows · `installer`

## Connections

- App entry → API router: starts · Application entry composes the API service
- API router → Core: uses · Shared runtime concerns are applied across routes
- API router → Audit: logs · Request and action tracing flows through audit helpers
- API router → DB layer: persists · Business modules rely on database and search abstractions
- API router → System: mounts · Identity and platform endpoints are part of the service
- API router → Datasource: mounts · Datasource management endpoints are part of the service
- API router → Chat: mounts · Chat lifecycle endpoints are part of the service
- API router → Dashboard: mounts · Dashboard endpoints are part of the service
- API router → Terminology: mounts · Terminology endpoints support query refinement
- API router → Training: mounts · Training-feedback endpoints support improvement loops
- API router → MCP: exposes · Tool-calling integration is exposed through the backend
- System → Auth middleware: enforces · Authentication middleware protects system-wide access
- Datasource → Embeddings: indexes · Datasource content is embedded for retrieval support
- Chat → AI models: calls · Chat orchestration delegates model calls to provider abstractions
- Chat → Templates: prompts · Chat execution depends on prompt and generation templates
- Templates → Datasource: selects · Templates include datasource selection and filtering logic
- Templates → Terminology: uses · Query interpretation can incorporate terminology assets
- Templates → AI models: formats · Prompt generation targets provider-specific model capabilities
- SPA shell → Chat UI: renders · Primary SPA surface includes conversational analytics
- SPA shell → Dashboard UI: renders · Primary SPA surface includes dashboard editing and preview
- SPA shell → Datasource UI: renders · Primary SPA surface includes datasource administration
- SPA shell → System UI: renders · Primary SPA surface includes system administration
- SPA shell → Embedded UI: renders · Primary SPA surface includes embeddable views
- Chat UI → Chart SSR: uses · Chart views can rely on server-side rendering support
- Dashboard UI → Chart SSR: uses · Dashboard composition and preview benefit from SSR charting
- Deploy → App entry: packages · Container and installer flows deliver the backend runtime
- Deploy → SPA shell: packages · Deployment bundles the SPA frontend assets
- Deploy → Chart SSR: packages · Deployment includes the chart rendering service
