# finos/perspective architecture

> Architecture diagram of the GitHub repository finos/perspective, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/finos/perspective
- Repository: https://github.com/finos/perspective

## Overview

##### Project Type and Purpose
- Perspective is essentially a data analytics and visualization engine that includes a low-level high-performance query engine (developed primarily in C++ and Rust) along with rich, framework‑agnostic UI components (packaged as Custom Elements for browsers and integrated with JupyterLab).
- The README emphasizes its use for large/streaming datasets, interactive reporting, dashboards, and integration with multiple client APIs (JavaScript, Python, Rust).

##### Major Components
a. Core Query Engine:
- The engine is implemented in C++ (within the cpp/ folder) and Rust (within multiple rust/ subdirectories, e.g., perspective-client, perspective-js, perspective-server, perspective-viewer, and the root perspective package).

b. API and Bindings Layer:
- JavaScript/TypeScript API: Provided as packages such as @finos/perspective, perspective-viewer, and perspective-viewer-d3fc with extensive code in the packages/ directory and the rust/perspective-js project.
- Python API: Found in perspective-python, which includes bindings, server handlers, and JupyterLab integrations.
- Rust API: The perspective crate exposes powerful Rust interfaces for server and client implementations.
- CLI Tool: There’s a perspective-cli package for command-line operations.

c. User Interface Components:
- Custom Elements-based UI is provided, which work in the browser either via WebAssembly or via a WebSocket server. The UI packages include perspective-viewer and its variants (e.g., perspective-viewer-datagrid, perspective-viewer-d3fc, perspective-viewer-openlayers, perspective-workspace).
- These packages organize HTML, CSS (LESS) and client-side JavaScript/TypeScript code to render interactive views.

d. Build, Testing, and Deployment:
- Build configurations for cargo (Cargo.toml files), cmake directories for C++ parts, and package.json files for Node-based builds are present.
- GitHub actions, CI workflows, Docker/devcontainer configurations, and extensive test suites (located across tests/ and tools/) indicate a mature development lifecycle.

##### Relationships and Data Flow
- Data ingestion starts at the core engine where large datasets are processed. Data flows through the query engine (in C++/Rust) and then is exposed via API layers.
- The API layers (Rust, Python, or JavaScript) allow client applications and UIs to interact with the engine either in-browser (via WebAssembly) or via a server (via WebSocket) mode.
- The UI component (viewer) subscribes to data, applies interactive configurations (pivot, filter, sort, etc.) provided via the API, and renders results.
- The CLI and integration packages (like perspective-jupyterlab) tie into this system enabling usage in a variety of deployment scenarios.

##### Architectural Patterns
- Layered Architecture: Clearly separated layers (engine, APIs, UI) make the system modular.
- Polyglot Environment: Multiple stacks (Rust, C++, Python, JavaScript) interoperate, usually bridged via WebAssembly or language-specific bindings.
- Client-Server Model: The engine can run in a server mode (e.g., using Node/Python/Rust servers) or in-browser (via WebAssembly), promoting flexibility.

- Clear labels for each major component (e.g., Query Engine, Rust API, Python API, JavaScript Client, UI Components, CLI, Build Infrastructure).

## Diagram (Mermaid)

```mermaid
graph TD
    %% External Dependencies
    ext1["Apache Arrow"]:::external
    ext2["ExprTK"]:::external

    %% Core Query Engine (C++ based)
    subgraph "Core Query Engine"
        ce1["Query Engine (C++)"]:::backend
        click ce1 "https://github.com/finos/perspective/tree/master/cpp/perspective"
    end

    %% Core Query Engine (Rust Components)
    subgraph "Rust Components"
        r1["Rust: perspective"]:::backend
        click r1 "https://github.com/finos/perspective/tree/master/rust/perspective"
        r2["Rust: perspective-server"]:::backend
        click r2 "https://github.com/finos/perspective/tree/master/rust/perspective-server"
        r3["Rust: perspective-client"]:::backend
        click r3 "https://github.com/finos/perspective/tree/master/rust/perspective-client"
        r4["Rust: perspective-js"]:::backend
        click r4 "https://github.com/finos/perspective/tree/master/rust/perspective-js"
        r5["Rust: perspective-python"]:::backend
        click r5 "https://github.com/finos/perspective/tree/master/rust/perspective-python"
    end

    %% API and UI Layers
    subgraph "API & UI Layers"
        js1["JS/TS API & UI: Perspective Viewer D3FC"]:::frontend
        click js1 "https://github.com/finos/perspective/tree/master/packages/perspective-viewer-d3fc"
        js2["JS/TS API & UI: Perspective Viewer Datagrid"]:::frontend
        click js2 "https://github.com/finos/perspective/tree/master/packages/perspective-viewer-datagrid"
        js3["JS/TS API & UI: Perspective Viewer OpenLayers"]:::frontend
        click js3 "https://github.com/finos/perspective/tree/master/packages/perspective-viewer-openlayers"
        js4["JS/TS API & UI: Perspective Workspace"]:::frontend
        click js4 "https://github.com/finos/perspective/tree/master/packages/perspective-workspace"
        py1["Python API & JupyterLab Integration"]:::frontend
        click py1 "https://github.com/finos/perspective/tree/master/packages/perspective-jupyterlab"
    end

    %% Integration Tools
    subgraph "Integration Tools"
        cli1["CLI Tool"]:::integration
        click cli1 "https://github.com/finos/perspective/tree/master/packages/perspective-cli"
    end

    %% Build, Testing, and Deployment Infrastructure
    subgraph "Build & Deployment"
        bd1["CMake & Build Configs"]:::infrastructure
        click bd1 "https://github.com/finos/perspective/tree/master/cmake"
        bd2["CI/CD & GitHub Actions"]:::infrastructure
        click bd2 "https://github.com/finos/perspective/blob/master/.github"
        bd3["Tools & Testing Utilities"]:::infrastructure
        click bd3 "https://github.com/finos/perspective/tree/master/tools"
    end

    %% Data Flow Connections
    ext1 --> ce1
    ext2 --> ce1

    ce1 -->|"results"| r1
    r1 -->|"processes"| r2
    r1 -->|"processes"| r3
    r1 -->|"processes"| r4
    r1 -->|"processes"| r5

    %% Core engine feeding API & UI Layers
    ce1 -->|"data"| js1
    ce1 -->|"data"| js2
    ce1 -->|"data"| js3
    ce1 -->|"data"| js4
    ce1 -->|"data"| py1

    %% Rust components also support API layers
    r2 -->|"API serving"| js1
    r3 -->|"bridges"| js2
    r4 -->|"supports"| js3
    r1 -->|"integration"| js4
    r5 -->|"bindings"| py1

    %% CLI interacting with Rust Server
    cli1 -->|"commands"| r2

    %% Build & Deployment infrastructure connected to core components
    bd1 --> ce1
    bd1 --> r1
    bd2 --> cli1
    bd3 --> ce1

    %% Styling definitions
    classDef backend fill:#aec6cf,stroke:#000,stroke-width:2px;
    classDef frontend fill:#c3fdb8,stroke:#000,stroke-width:2px;
    classDef integration fill:#fdfd96,stroke:#000,stroke-width:2px;
    classDef infrastructure fill:#ffb347,stroke:#000,stroke-width:2px;
    classDef external fill:#d3d3d3,stroke:#000,stroke-width:2px;
```
