# finos/timebase-ce architecture

> Architecture diagram of the GitHub repository finos/timebase-ce, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/finos/timebase-ce
- Repository: https://github.com/finos/timebase-ce

## Overview

##### Big Picture
- The project is an enterprise‐grade, open‑source time‑series data management and messaging system used in financial services.
- It functions both as a time‑series database (storing and querying historical data) and as a high‑performance real‑time data streaming/messaging server.
- Its architecture needs to support low‐latency data ingestion, high throughput data transmission, and real‑time querying, while also exposing APIs (including JDBC, REST, websockets) and integration modules (e.g., cloud/Docker support, S3 clients, SNMP agents, web monitoring tools).

##### High-Level Components
- TimeBase Server Core:
– This is the heart of the system and is subdivided into subcomponents such as:
- Data Ingestion Layer: accepts incoming time-series messages—via native protocols (IPC/UDP), REST endpoints, and possibly other messaging APIs.
- Storage Engine (“TickDB”): responsible for persistent data storage, including mechanisms for organizing “streams” of time-series data in a resilient and scalable way. It also supports historical queries.
- Query Processing Engine: supports both SQL-style and low‑latency, real‑time stream consumption.
- Messaging/Streaming Infrastructure: handles publication and subscription to “topics” or streams. This part can integrate high‑performance messaging channels (for example, using Aeron) and low‑latency APIs.
- API and Connector Modules:
– JDBC Module: for SQL access.
– Client Libraries and SDKs for various integrations.
– REST/WS interfaces for external integration.
- Web Layers and Monitoring:
– Web Monitor: the webmonitor and web modules include a UI for system status (e.g., connections, active sessions, server metrics, logs).
– SNMP/Admin modules: there’s an SNMP agent component to expose operational metrics.
- Build & Deployment Support:
– There are Gradle-based build modules and distribution scripts for Docker, Kubernetes, and cloud deployment.

##### Architectural Patterns and Relationships
- The system follows a modular, component‑oriented design. Each module (server, client, pub, jdbc, etc.) is built as an independent Gradle sub‑project.
- Data ingress (multiple protocols) and differentiate between the persistent storage (“TickDB”) and the real‑time stream distribution.
- That Messaging components and the Query engine can work concurrently, with data “streams” having user-defined schemas.
- Integration points (SNMP agents, REST endpoints, JDBC API, and web UIs) that interface with external applications.
– Data flows from external data providers into the ingestion layer.
– Ingestion flows into the core storage engine and is “published” via the messaging system.
– Queries and JDBC calls being routed to the Query Processing Engine.
– Administration / monitoring calls going to the SNMP agent, web UI, and REST layer.

##### Technologies and Frameworks
- Java (11+): the whole system is built in Java with Gradle as the build tool.
- Gradle builds and buildSrc for custom build plugins.
- Cloud/Container orchestration via Docker and Kubernetes (as per provided Docker-related files).
- Low-level networking libraries, e.g., Aeron for high‑performance messaging.
- Complementary languages: There are CPP utilities integrated with TimeBase for performance‑sensitive use cases.
- API endpoints expose the system via REST endpoints and websockets.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Main System Container
    subgraph "TimeBase Community Edition"
        %% External Data Provider
        externalProviders["External Data Providers"]:::external

        %% Core Server and Engine Components
        subgraph "Core Server (TimeBase Server Core)"
            coreServer["Core Server (TimeBase Server Core)"]:::service
            ingestion["Data Ingestion Layer"]:::service
            tickdb["Storage Engine (TickDB)"]:::database
            queryEngine["Query Processing Engine"]:::service
            messaging["Messaging/Streaming Engine"]:::service
        end

        %% API Modules
        subgraph "API Modules"
            jdbc["JDBC Connector"]:::service
            rest["REST/WS API"]:::service
            client["Client Libraries & Tools"]:::service
        end

        %% Monitoring & Administration
        subgraph "Monitoring & Administration"
            snmp["SNMP Agent"]:::admin
            webMonitor["Web Monitor & Administration UI"]:::admin
        end

        %% Deployment & Integration
        subgraph "Deployment & Integration"
            cloud["Cloud/Docker/Kubernetes Support"]:::deployment
            s3["S3 Integration Modules"]:::deployment
        end

        %% Build & Deployment Infrastructure
        build["Build & Deployment Infrastructure"]:::build
    end

    %% Data Flow Arrows
    externalProviders -->|"MessageIngestion"| ingestion
    ingestion -->|"StoresData"| tickdb
    ingestion -->|"RealTimeStream"| messaging

    %% API to Query Processing Flow
    jdbc -->|"SQLQuery"| queryEngine
    rest -->|"APIQuery"| queryEngine
    client -->|"ClientQuery"| queryEngine

    %% Query Request Flow from Storage to Query Engine
    tickdb -->|"QueryRequests"| queryEngine

    %% Administration Flows
    queryEngine -->|"AdminMonitoring"| snmp
    tickdb -->|"AdminMonitoring"| snmp
    messaging -->|"HealthMetrics"| webMonitor

    %% Deployment Integration Flows
    ingestion -->|"DeploymentIntegration"| cloud
    tickdb -->|"DeploymentIntegration"| cloud
    messaging -->|"DeploymentIntegration"| cloud
    s3 -->|"DataImportExport"| tickdb

    %% Build Infrastructure Links
    build ---|"Gradle&Docker"| coreServer
    build ---|"Gradle&Docker"| s3

    %% Legend
    subgraph "Legend"
        legend1["Rectangle: Service"]:::service
        legend2["Cylinder: Database"]:::database
        legend3["Cloud: External"]:::external
        legend4["Hexagon: Administration"]:::admin
        legend5["Parallelogram: Deployment"]:::deployment
        legend6["Note: Build Infrastructure"]:::build
    end

    %% Click Events
    click coreServer "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/server"
    click tickdb "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/server/src/main/java/com/epam/deltix/qsrv/hf/tickdb"
    click queryEngine "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/api/src/main/java/com/epam/deltix/qsrv/hf/tickdb/lang"
    click messaging "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/pub"
    click jdbc "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/jdbc"
    click s3 "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/s3"
    click webMonitor "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/webmonitor"
    click client "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/client"
    click build "https://github.com/finos/timebase-ce/tree/main-6.1/build.gradle, gradle/"
    click snmp "https://github.com/finos/timebase-ce/tree/main-6.1/java/timebase/snmpagent"

    %% Styles
    classDef service fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef database fill:#F9E79F,stroke:#7D6608,stroke-width:2px;
    classDef admin fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef deployment fill:#ABEBC6,stroke:#196F3D,stroke-width:2px;
    classDef external fill:#D2B4DE,stroke:#512E5F,stroke-width:2px;
    classDef build fill:#FAD7A0,stroke:#B9770E,stroke-width:2px;
```
