How finos/perspective works
Project Type and Purpose
• Perspective is essentially a data analytics and visualization engine that includes a low-level high-performance query engine (developed primarily in C++ and Rust) along with rich, framework‑agnostic UI components (packaged as Custom Elements for browsers and integrated with JupyterLab).
• The README emphasizes its use for large/streaming datasets, interactive reporting, dashboards, and integration with multiple client APIs (JavaScript, Python, Rust).
Major Components
a. Core Query Engine:
• The engine is implemented in C++ (within the cpp/ folder) and Rust (within multiple rust/ subdirectories, e.g., perspective-client, perspective-js, perspective-server, perspective-viewer, and the root perspective package).
b. API and Bindings Layer:
• JavaScript/TypeScript API: Provided as packages such as @finos/perspective, perspective-viewer, and perspective-viewer-d3fc with extensive code in the packages/ directory and the rust/perspective-js project.
• Python API: Found in perspective-python, which includes bindings, server handlers, and JupyterLab integrations.
• Rust API: The perspective crate exposes powerful Rust interfaces for server and client implementations.
• CLI Tool: There’s a perspective-cli package for command-line operations.
c. User Interface Components:
• Custom Elements-based UI is provided, which work in the browser either via WebAssembly or via a WebSocket server. The UI packages include perspective-viewer and its variants (e.g., perspective-viewer-datagrid, perspective-viewer-d3fc, perspective-viewer-openlayers, perspective-workspace).
• These packages organize HTML, CSS (LESS) and client-side JavaScript/TypeScript code to render interactive views.
d. Build, Testing, and Deployment:
• Build configurations for cargo (Cargo.toml files), cmake directories for C++ parts, and package.json files for Node-based builds are present.
• GitHub actions, CI workflows, Docker/devcontainer configurations, and extensive test suites (located across tests/ and tools/) indicate a mature development lifecycle.
Relationships and Data Flow
• Data ingestion starts at the core engine where large datasets are processed. Data flows through the query engine (in C++/Rust) and then is exposed via API layers.
• The API layers (Rust, Python, or JavaScript) allow client applications and UIs to interact with the engine either in-browser (via WebAssembly) or via a server (via WebSocket) mode.
• The UI component (viewer) subscribes to data, applies interactive configurations (pivot, filter, sort, etc.) provided via the API, and renders results.
• The CLI and integration packages (like perspective-jupyterlab) tie into this system enabling usage in a variety of deployment scenarios.
Architectural Patterns
• Layered Architecture: Clearly separated layers (engine, APIs, UI) make the system modular.
• Polyglot Environment: Multiple stacks (Rust, C++, Python, JavaScript) interoperate, usually bridged via WebAssembly or language-specific bindings.
• Client-Server Model: The engine can run in a server mode (e.g., using Node/Python/Rust servers) or in-browser (via WebAssembly), promoting flexibility.
• Clear labels for each major component (e.g., Query Engine, Rust API, Python API, JavaScript Client, UI Components, CLI, Build Infrastructure).