How tencentmusic/supersonic works
1. Repository shape
- This is a Maven multi-module monorepo with three major product surfaces: Headless BI, Chat BI, and Auth, plus shared libraries, launchers, and a separate webapp workspace.
- The Java backend is split by domain boundaries rather than by technical layers alone, and the frontend is split into an admin-style app and an embeddable chat SDK.
2. Top-level runtime entrypoints
- launchers/headless, launchers/chat, and launchers/standalone are the packaging/runtime entrypoints.
- standalone is the demo distribution that bundles the semantic layer and chat flows together with sample configs, H2/MySQL/Postgres SQL seeds, HanLP dictionaries, and OpenSearch index templates.
- Docker and CI workflows indicate the project is intended to run as containerized services as well as local Java processes.
3. Shared foundation: common/
- common contains the cross-cutting infrastructure used by both Headless and Chat:
- SQL parsing and rewriting utilities based on Calcite and JSqlParser.
- Configuration, threading, logging, JSON, HTTP, encryption, retry, and context utilities.
- Shared domain objects for chat models, embeddings, semantic/query abstractions, and return codes.
- Model/provider integrations for LLMs and embeddings, including OpenAI, LocalAI, Dify, Ollama, in-memory, and vector store adapters.
- This module is the main boundary for reusable platform behavior.
4. Headless BI core (headless/core)
- This is the semantic-query engine: it translates structured or semantic requests into executable SQL.
- Key subsystems:
- DB adaptors for dialect-specific behavior across MySQL, Postgres, ClickHouse, Trino, Presto, Oracle, HANA, H2, Kyuubi, DuckDB, and StarRocks.
- Query executor and accelerator layer for executing and optimizing generated SQL.
- Translator/parser layer that turns semantic objects into SQL, with optimizer stages like dialect adjustment and result limiting.
- Cache for query results and semantic computation reuse.
- This module is the core boundary between semantic intent and physical data access.
5. Headless BI server (headless/server)
- This is the product-facing semantic layer service: REST controllers, application services, repositories, and MyBatis persistence.
- It owns the metadata lifecycle for apps, databases, datasets, models, metrics, dimensions, tags, query rules, dictionaries, and stats.
- It also includes:
- listeners for embedding and dictionary refreshes,
- scheduled tasks for dictionary reload and meta embedding,
- YAML/model builders and converters that map persisted metadata into semantic structures.
- In architecture terms, this is the authoritative metadata and governance service for the semantic model.
6. Chat BI backend (chat/server)
- This is the conversational query pipeline.
- The flow is organized as:
- REST controllers receive chat/config/plugin/memory requests.
- Services coordinate chat state, query parsing, execution, configuration, statistics, and agent/plugin management.
- Parsers choose between rule-based, plain text, NL2SQL, and NL2Plugin paths.
- Executors run SQL, plugin calls, or plain-text responses.
- Processors post-process parse and execution results for formatting, recommendation, metric logic, and interpretation.
- This module is the orchestration layer that turns user conversation into actionable query results.
7. Chat knowledge, memory, and plugin subsystems
- chat/server is not just a chat UI backend; it has its own internal knowledge pipeline.
- Knowledge / schema mapping: builds searchable representations of datasets, dimensions, metrics, values, and aliases; supports embedding recall and keyword/HanLP-based matching.
- Memory: stores historical conversational trajectories and review status for reuse in prompting and retrieval.
- Plugins: modelled as first-class tools with managers, recognizers, events, and tool configs; they let an LLM route a query to a third-party capability instead of SQL.
- These are the key AI-facing extension points on the Chat side.
8. Authentication and authorization
- auth/api defines the shared auth contracts for user and access-control operations.
- auth/authentication implements user identity retrieval, token handling, interceptors, and strategy selection (for example header-based or fake user strategies).
- auth/authorization implements group and resource authorization APIs.
- The repo’s auth boundary is separate from the semantic/chat domains, but its outputs feed the rest of the platform through request context and permission checks.
9. Web applications (webapp/)
- supersonic-fe is the main management console:
- semantic model authoring,
- datasets, metrics, dimensions, domains, tags, query rules, system settings,
- login and user administration,
- graph/canvas-style modeling pages.
- chat-sdk is a reusable embeddable chat component library with conversation views, result renderers, charts, and recommendation UI.
- This split suggests two distinct frontend consumers: operators/engineers in the admin app, and end-users/integrators through the SDK.
10. Data stores and external integrations
- Persistence is relational and ORM-based, using MyBatis mappers plus repository/service layers.
- The repo also includes OpenSearch resources for chat memory and meta collections, and vector-store abstractions for embedding retrieval.
- External AI services and local models are pluggable rather than hard-coded, which is important to the architecture: the system can swap LLM providers, embedding models, and retrieval stores without changing the core query flow.
11. End-to-end data flow
- In Headless BI, metadata is curated in the semantic layer, persisted, and exposed through APIs; the core translator then converts semantic requests into dialect-aware SQL for the target database.
- In Chat BI, user text enters the chat backend, is mapped against semantic knowledge and memory, parsed into semantic intent, corrected if needed, translated to SQL or routed to a plugin, then executed and formatted back to the UI.
- The two halves are intentionally coupled through the semantic model: chat depends on headless-curated metadata for grounding, and headless benefits from chat-generated access patterns and recommendations.
12. Operational and build boundaries
- CI/CD, Docker, assembly scripts, and launcher modules indicate the project is designed to ship as a packaged Java platform with multiple deployable profiles.
- Test coverage is split by module and includes parser, translator, semantic modeling, and service-level tests, reflecting the system’s emphasis on correctness in SQL generation and metadata handling.
Main entry points: chat/server Chat server · headless/server Headless server · common Common foundation · api Auth API
Components 16
Runtime & packaging
Deployable launchers, assembly, Docker, and build scripts.
- Headless launcher · Packaging/runtime entrypoint for the headless BI service.
launchers/headless - Chat launcher · Packaging/runtime entrypoint for the chat BI service.
launchers/chat - Standalone demo · Bundled demo runtime with semantic, chat, sample data, and search resources.
launchers/standalone - Launcher common · Shared Spring/web configuration used by launcher modules.
launchers/common
Backend platform
Shared backend modules for auth, common services, semantic BI, and chat BI.
- Common foundation · Cross-cutting utilities, shared domain objects, model/provider adapters, SQL parsing, and embedding store abstractions.
common - Auth API · Shared authentication and authorization interfaces, requests, responses, and context helpers.
auth/api - Authn service · User identity, token handling, interceptors, and strategy-based authentication implementation.
auth/authentication - Authz service · Group and resource authorization service.
auth/authorization - Headless API · Semantic-layer request/response model for apps, datasets, metrics, dimensions, rules, and queries.
headless/api - Semantic core · Dialect-aware semantic query translation, DB adaptors, query execution, optimization, and cache.
headless/core - Headless server · Authoritative semantic metadata service with REST APIs, persistence, builders, listeners, and scheduled refresh tasks.
headless/server - Headless chat · Semantic chat pipeline for knowledge mapping, parsing, correction, and query construction on top of the headless model.
headless/chat - Chat API · Chat requests, responses, memory states, recommendations, and parsing contracts.
chat/api - Chat server · Conversational orchestration layer with controllers, services, parsers, executors, processors, memory, knowledge, and plugins.
chat/server
Web apps
Management console and embeddable chat SDK.
- Supersonic FE · Management console for semantic modeling, permissions, login, system settings, and graph-based authoring.
webapp/packages/supersonic-fe - Chat SDK · Reusable chat component library for conversation, results, charts, recommendations, and agent UI.
webapp/packages/chat-sdk
Connections 33
- Headless launcher → Headless server: starts · Bootstraps the headless semantic service.
- Headless launcher → Semantic core: packages · Includes the semantic translation engine.
- Headless launcher → Authn service: uses · Wires authentication into the service runtime.
- Headless launcher → Authz service: uses · Wires authorization checks into the service runtime.
- Chat launcher → Chat server: starts · Bootstraps the conversational service.
- Chat launcher → Headless server: grounds on · Chat depends on curated semantic metadata.
- Chat launcher → Authn service: uses · Wires authentication into the service runtime.
- Standalone demo → Headless server: bundles · Demo distribution includes semantic layer resources.
- Standalone demo → Chat server: bundles · Demo distribution includes chat flows and config.
- Standalone demo → Headless chat: bundles · Includes headless chat and semantic-query demos.
- Standalone demo → Common foundation: bundles · Ships shared platform runtime and resources.
- Launcher common → Common foundation: configures · Provides shared Spring/web wiring for launchers.
- Headless server → Headless API: implements · Serves the semantic-layer contract.
- Headless server → Semantic core: delegates to · Uses semantic translation and execution core.
- Headless server → Common foundation: depends on · Uses shared utilities, models, and integrations.
- Headless chat → Headless API: consumes · Builds semantic queries from headless contracts.
- Headless chat → Semantic core: executes via · Turns semantic intent into database SQL.
- Headless chat → Common foundation: depends on · Uses shared SQL, LLM, and retrieval infrastructure.
- Chat server → Chat API: implements · Serves the conversational contract.
- Chat server → Headless server: reads metadata · Relies on semantic model, datasets, and dictionaries.
- Chat server → Common foundation: depends on · Uses shared models, tools, and model provider adapters.
- Chat server → Auth API: uses · Consumes user and permission context.
- Chat server → Authn service: uses · Depends on authenticated user identity.
- Chat server → Authz service: uses · Applies access-control checks to chat operations.
- Authn service → Auth API: implements · Provides identity and token services for the auth contract.
- Authz service → Auth API: implements · Provides authorization services for the auth contract.
- Supersonic FE → Headless server: calls · Management console drives semantic metadata APIs.
- Supersonic FE → Auth API: authenticates via · Login and user administration flow through auth APIs.
- Chat SDK → Chat server: calls · Embeddable UI consumes chat and recommendation APIs.
- Chat SDK → Headless server: reads model data · Renders semantic/query results grounded in headless metadata.
- Common foundation → Auth API: shares context with · Shared user, permission, and return-code types.
- Common foundation → Semantic core: supplies · Provides SQL parsing, dialect, and model-provider utilities.
- Common foundation → Chat server: supplies · Provides embeddings, LLM adapters, and retrieval utilities.