# apache/polaris architecture

> Architecture diagram of the GitHub repository apache/polaris, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/polaris
- Repository: https://github.com/apache/polaris

## Overview

##### Main Components
- Polaris Core – Represent the core business logic, entity definitions, and persistence mechanisms. This includes modules such as “polaris-core.”
- API Modules –
– The Iceberg REST Service (“polaris-api-iceberg-service”) for the Iceberg REST API.
– The Management API (“polaris-api-management-service” and “polaris-api-management-model”) for administration and catalog management.
- Service Modules – The common service components found in “polaris-service-common,” which coordinate interactions between APIs and back-end persistence.
- Quarkus Runtime Modules – These include “polaris-quarkus-service”, “polaris-quarkus-defaults”, “polaris-quarkus-server”, and “polaris-quarkus-admin-tool.” They provide the runtime environment along with configuration defaults and administrative tooling.
- Persistence Modules – Components such as “polaris-jpa-model” and “polaris-eclipselink” handle the MetaStoreManager interface and actual persistence of catalog metadata.
- Client & Plugins – There is also a distinct client folder (with a Python client CLI) and additional plugins (e.g., Spark integration) that extend or integrate with external tools.
- Infrastructure & Deployment – Include Dockerfiles, Helm charts (in the helm directory) for Kubernetes deployment, Kind deployment scripts in the “getting-started” directory, and related orchestration files.

##### Out the Relationships and Interactions
- That API components expose REST endpoints (both for Iceberg and management functions) and that these endpoints call into the service modules.
- How the service modules interact with the core logic in “polaris-core” for business rules and then delegate tasks to the persistence modules for storage.
- Quarkus runtime modules as the container and configuration layer—which host the API endpoints and provide operational capabilities (such as monitoring via health/metrics endpoints).
- Supporting infrastructure (like Continuous Integration workflows from the “.github/workflows” directory, docker-compose setups, and Helm/Kubernetes deployment resources).

##### Architectural Patterns and Design Principles
- Modularization: The separation into API, core, service, runtime, and persistence modules illustrates a clear separation of concerns.
- Interoperability: Emphasize the REST interface that allows multiple external engines (Spark, Trino, etc.) to connect.
- Containerization & Deployment: Recognize the use of Docker, Kubernetes (Helm charts), and integration testing; these are integral to how Polaris is built, deployed, and maintained.

##### Specific Technologies, Frameworks, and Libraries
- Java 21+ and Gradle – Mark these as your build and execution environment.
- Quarkus – Noting its use for dependency injection, configuration, and fast startup in cloud-native environments.
- Eclipselink and JPA – For persistence and ORM.
- Docker and Kubernetes (via Helm charts) – Representing the container build and deployment pipeline.
- REST/JSON API – Indicate that the system leverages standardized REST endpoints for inter-service communications.
- OpenAPI specifications – The API definitions provided in the “spec” directory should be mentioned as they drive integration and client code generation.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Clients
    subgraph "Client & Plugins Layer"
        CLI["Python Client CLI"]:::external
        SPARK["Spark Integration Plugin"]:::external
    end

    %% Quarkus Runtime Modules (Deployment Environment)
    subgraph "Quarkus Runtime Modules"
        Q_SERVICE["Polaris Quarkus Service"]:::quarkus
        Q_DEFAULTS["Polaris Quarkus Defaults"]:::quarkus
        Q_SERVER["Polaris Quarkus Server"]:::quarkus
        Q_ADMIN["Polaris Quarkus Admin Tool"]:::quarkus
    end

    %% API Modules hosted on Quarkus
    subgraph "API Modules"
        ICEBERG["Iceberg REST Service"]:::api
        MGMT_SERVICE["Management API Service"]:::api
        MGMT_MODEL["Management API Model"]:::api
    end

    %% Service Layer
    COMMON["Common Service Logic"]:::service

    %% Polaris Core
    CORE["Polaris Core"]:::core

    %% Persistence Modules
    subgraph "Persistence Modules"
        JPA["JPA Model"]:::persistence
        ECLIPSE["EclipseLink Persistence"]:::persistence
    end

    %% Infrastructure & Deployment
    subgraph "Infra & Deployment"
        HELM["Helm Charts"]:::infra
        KIND["Kind Deployment Scripts"]:::infra
        CICD["CI/CD Workflows"]:::infra
    end

    %% Connections
    %% External Clients call API Modules
    CLI -->|"calls"| ICEBERG
    CLI -->|"calls"| MGMT_SERVICE
    SPARK -->|"triggers"| ICEBERG

    %% Quarkus Runtime Modules deploy API Modules
    Q_SERVICE --- ICEBERG
    Q_DEFAULTS --- ICEBERG
    Q_SERVER --- MGMT_SERVICE
    Q_ADMIN --- MGMT_SERVICE
    Q_SERVICE --- MGMT_MODEL

    %% API Modules interact with Service Logic
    ICEBERG -->|"invokes"| COMMON
    MGMT_SERVICE -->|"invokes"| COMMON
    MGMT_MODEL -->|"supports"| COMMON

    %% Service Layer to Polaris Core
    COMMON -->|"delegates to"| CORE

    %% Polaris Core uses Persistence Modules
    CORE -->|"stores/retrieves data"| JPA
    CORE -->|"stores/retrieves data"| ECLIPSE

    %% Infra deploys Quarkus Runtime
    HELM -->|"deploys"| Q_SERVICE
    KIND -->|"deploys"| Q_SERVICE
    CICD -->|"orchestrates"| Q_SERVICE

    %% Click Events
    click CORE "https://github.com/apache/polaris/tree/main/polaris-core"
    click ICEBERG "https://github.com/apache/polaris/tree/main/api/iceberg-service"
    click MGMT_SERVICE "https://github.com/apache/polaris/tree/main/api/management-service"
    click MGMT_MODEL "https://github.com/apache/polaris/tree/main/api/management-model"
    click COMMON "https://github.com/apache/polaris/tree/main/service/common"
    click Q_SERVICE "https://github.com/apache/polaris/tree/main/quarkus/service"
    click Q_DEFAULTS "https://github.com/apache/polaris/tree/main/quarkus/defaults"
    click Q_SERVER "https://github.com/apache/polaris/tree/main/quarkus/server"
    click Q_ADMIN "https://github.com/apache/polaris/tree/main/quarkus/admin"
    click JPA "https://github.com/apache/polaris/tree/main/extension/persistence/jpa-model"
    click ECLIPSE "https://github.com/apache/polaris/tree/main/extension/persistence/eclipselink"
    click CLI "https://github.com/apache/polaris/tree/main/client/python"
    click SPARK "https://github.com/apache/polaris/tree/main/plugins/spark"
    click HELM "https://github.com/apache/polaris/tree/main/helm"
    click KIND "https://github.com/apache/polaris/tree/main/getting-started"
    click CICD "https://github.com/apache/polaris/tree/main/.github/workflows"

    %% Styles
    classDef external fill:#ffeb3b,stroke:#333,stroke-width:2px;
    classDef api fill:#ff9800,stroke:#333,stroke-width:2px;
    classDef service fill:#2196f3,stroke:#fff,stroke-width:2px;
    classDef core fill:#f44336,stroke:#fff,stroke-width:2px;
    classDef persistence fill:#4caf50,stroke:#fff,stroke-width:2px;
    classDef quarkus fill:#9c27b0,stroke:#fff,stroke-width:2px;
    classDef infra fill:#607d8b,stroke:#fff,stroke-width:2px;
```
