# dimitrijejankov/plinycompute architecture

> Architecture diagram of the GitHub repository dimitrijejankov/plinycompute, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dimitrijejankov/plinycompute
- Repository: https://github.com/dimitrijejankov/plinycompute

## Overview

PlinyCompute is a C++ platform for building and running distributed data and computation workloads in standalone or distributed clusters. A user submits computations and manages datasets through client APIs; computation planning and execution run on the platform, while storage and catalog services manage the data and its metadata. The tree also contains representative machine-learning and linear-algebra workloads. The README documents broad capabilities, but the excerpts do not expose every server-side dispatch path, so some documented workflow links are shown without claiming implementation details.

Main entry points: [`pdb/src/client/source/PDBClient.cc`](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/PDBClient.cc) (PDB client), [`pdb/src/pipeline/source/pipeline/Pipeline.cc`](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/pipeline/source/pipeline/Pipeline.cc) (Execution pipeline), [`applications/TestGMM/TestGmmLazy.cc`](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestGMM/TestGmmLazy.cc) (GMM workload), [`pdb/src/client/headers/PDBCatalogClient.h`](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/headers/PDBCatalogClient.h) (Catalog client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Client and query APIs"]
  node_client["PDB client<br/>[PDBClient.cc]"]
  node_catalog_client["Catalog client<br/>[PDBCatalogClient.h]"]
  node_storage_client["Storage client"]
  node_query_analyzer["Query analyzer"]
  node_lambda["Lambda expressions<br/>[Lambda.h]"]
  node_computation_client["Computation client"]
end

subgraph group_compute["Computation runtime"]
  node_comp_server["Computation server"]
  node_optimizer["Physical optimizer"]
  node_pipeline["Execution pipeline<br/>[Pipeline.cc]"]
  node_join_pipeline["Join pipeline"]
  node_object_model["Object model<br/>[Handle.h]"]
end

subgraph group_data["Data and metadata"]
  node_catalog_server["Catalog service<br/>[CatalogServer.cc]"]
  node_catalog_store[("Catalog metadata")]
  node_distributed_storage["Distributed storage"]
  node_storage_manager["Storage manager"]
  node_buffer_manager["Buffer manager"]
end

subgraph group_workloads["Example workloads"]
  node_gmm["GMM workload<br/>[TestGmmLazy.cc]"]
  node_gmm_model["GMM model<br/>[GmmModel.h]"]
  node_lda["LDA workload<br/>[TestLDA.cc]"]
  node_lda_join["LDA topic sampling"]
  node_linear_algebra["Linear algebra"]
  node_matrix_multiply["Matrix multiplication"]
end

node_user(("Application user"))

node_user -->|"submits work"| node_client
node_client -.->|"manages metadata"| node_catalog_client
node_client -.->|"stores data"| node_storage_client
node_client -.->|"runs computations"| node_computation_client
node_client -.->|"analyzes queries"| node_query_analyzer
node_computation_client -.->|"submits computation"| node_comp_server
node_comp_server -.->|"plans execution"| node_optimizer
node_optimizer -.->|"builds execution"| node_pipeline
node_pipeline -.->|"uses join pipeline"| node_join_pipeline
node_pipeline -->|"processes objects"| node_object_model
node_storage_client -->|"uses catalog"| node_catalog_client
node_distributed_storage -->|"uses catalog"| node_catalog_client
node_catalog_server -.->|"manages metadata"| node_catalog_store
node_storage_manager -->|"uses catalog"| node_catalog_server
node_storage_manager -.->|"manages pages"| node_buffer_manager
node_user -->|"runs example"| node_gmm
node_gmm -->|"submits computations"| node_client
node_gmm -->|"builds expressions"| node_lambda
node_gmm_model -->|"uses object types"| node_object_model
node_user -->|"runs example"| node_lda
node_lda -->|"uses client"| node_client
node_lda_join -.->|"creates assignments"| node_object_model
node_user -->|"uses library"| node_linear_algebra
node_user -.->|"runs example"| node_matrix_multiply

click node_client "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/PDBClient.cc"
click node_catalog_client "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/headers/PDBCatalogClient.h"
click node_storage_client "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/headers/PDBDistributedStorageClient.h"
click node_query_analyzer "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/QueryGraphAnalyzer.cc"
click node_lambda "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/lambdas/headers/Lambda.h"
click node_computation_client "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/PDBComputationClient.cc"
click node_comp_server "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/computationServer/sources/PDBComputationServerFrontend.cc"
click node_optimizer "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/computationServer/sources/physicalOptimizer/PDBAbstractPhysicalNode.cc"
click node_pipeline "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/pipeline/source/pipeline/Pipeline.cc"
click node_join_pipeline "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/pipeline/source/pipeline/JoinBroadcastPipeline.cc"
click node_object_model "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/objectModel/headers/Handle.h"
click node_catalog_server "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/catalog/source/CatalogServer.cc"
click node_catalog_store "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/catalog/headers/PDBCatalogStorage.h"
click node_distributed_storage "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/distributedStorage/source/PDBDistributedStorage.cc"
click node_storage_manager "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/storageManager/source/PDBStorageManagerFrontend.cc"
click node_buffer_manager "https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/bufferManager/source/PDBBufferManagerImpl.cc"
click node_gmm "https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestGMM/TestGmmLazy.cc"
click node_gmm_model "https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestGMM/sharedLibraries/headers/GmmModel.h"
click node_lda "https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestLDA/TestLDA.cc"
click node_lda_join "https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestLDA/sharedLibraries/headers/LDADocWordTopicJoin.h"
click node_linear_algebra "https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestLinearAlgebra/linearAlgebraDSL/source/LAParserHelperFunctions.cc"
click node_matrix_multiply "https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestMatrixMultiply/TestMatrixMultiply.cc"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_client,node_catalog_client,node_storage_client,node_query_analyzer,node_lambda,node_computation_client,node_user toneBlue
class node_comp_server,node_optimizer,node_pipeline,node_join_pipeline,node_object_model toneAmber
class node_catalog_server,node_catalog_store,node_distributed_storage,node_storage_manager,node_buffer_manager toneMint
class node_gmm,node_gmm_model,node_lda,node_lda_join,node_linear_algebra,node_matrix_multiply toneRose
```

## Components

### Client and query APIs

- [PDB client](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/PDBClient.cc): `pdb/src/client/source/PDBClient.cc`
- [Catalog client](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/headers/PDBCatalogClient.h): `pdb/src/client/headers/PDBCatalogClient.h`
- [Storage client](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/headers/PDBDistributedStorageClient.h): `pdb/src/client/headers/PDBDistributedStorageClient.h`
- [Query analyzer](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/QueryGraphAnalyzer.cc): `pdb/src/client/source/QueryGraphAnalyzer.cc`
- [Lambda expressions](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/lambdas/headers/Lambda.h): `pdb/src/lambdas/headers/Lambda.h`
- [Computation client](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/client/source/PDBComputationClient.cc): `pdb/src/client/source/PDBComputationClient.cc`

### Computation runtime

- [Computation server](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/computationServer/sources/PDBComputationServerFrontend.cc): `pdb/src/computationServer/sources/PDBComputationServerFrontend.cc`
- [Physical optimizer](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/computationServer/sources/physicalOptimizer/PDBAbstractPhysicalNode.cc): `pdb/src/computationServer/sources/physicalOptimizer/PDBAbstractPhysicalNode.cc`
- [Execution pipeline](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/pipeline/source/pipeline/Pipeline.cc): `pdb/src/pipeline/source/pipeline/Pipeline.cc`
- [Join pipeline](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/pipeline/source/pipeline/JoinBroadcastPipeline.cc): `pdb/src/pipeline/source/pipeline/JoinBroadcastPipeline.cc`
- [Object model](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/objectModel/headers/Handle.h): `pdb/src/objectModel/headers/Handle.h`

### Data and metadata

- [Catalog service](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/catalog/source/CatalogServer.cc): `pdb/src/catalog/source/CatalogServer.cc`
- [Catalog metadata](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/catalog/headers/PDBCatalogStorage.h): `pdb/src/catalog/headers/PDBCatalogStorage.h`
- [Distributed storage](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/distributedStorage/source/PDBDistributedStorage.cc): `pdb/src/distributedStorage/source/PDBDistributedStorage.cc`
- [Storage manager](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/storageManager/source/PDBStorageManagerFrontend.cc): `pdb/src/storageManager/source/PDBStorageManagerFrontend.cc`
- [Buffer manager](https://github.com/dimitrijejankov/plinycompute/blob/master/pdb/src/bufferManager/source/PDBBufferManagerImpl.cc): `pdb/src/bufferManager/source/PDBBufferManagerImpl.cc`

### Example workloads

- [GMM workload](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestGMM/TestGmmLazy.cc): `applications/TestGMM/TestGmmLazy.cc`
- [GMM model](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestGMM/sharedLibraries/headers/GmmModel.h): `applications/TestGMM/sharedLibraries/headers/GmmModel.h`
- [LDA workload](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestLDA/TestLDA.cc): `applications/TestLDA/TestLDA.cc`
- [LDA topic sampling](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestLDA/sharedLibraries/headers/LDADocWordTopicJoin.h): `applications/TestLDA/sharedLibraries/headers/LDADocWordTopicJoin.h`
- [Linear algebra](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestLinearAlgebra/linearAlgebraDSL/source/LAParserHelperFunctions.cc): `applications/TestLinearAlgebra/linearAlgebraDSL/source/LAParserHelperFunctions.cc`
- [Matrix multiplication](https://github.com/dimitrijejankov/plinycompute/blob/master/applications/TestMatrixMultiply/TestMatrixMultiply.cc): `applications/TestMatrixMultiply/TestMatrixMultiply.cc`

### Other components

- **Application user**

## Connections

- Application user → PDB client: submits work
- PDB client → Catalog client: manages metadata
- PDB client → Storage client: stores data
- PDB client → Computation client: runs computations
- PDB client → Query analyzer: analyzes queries
- Computation client → Computation server: submits computation
- Computation server → Physical optimizer: plans execution
- Physical optimizer → Execution pipeline: builds execution
- Execution pipeline → Join pipeline: uses join pipeline
- Execution pipeline → Object model: processes objects
- Storage client → Catalog client: uses catalog (evidence: `pdb/src/client/headers/PDBDistributedStorageClient.h`)
- Distributed storage → Catalog client: uses catalog (evidence: `pdb/src/distributedStorage/source/PDBDistributedStorage.cc`)
- Catalog service → Catalog metadata: manages metadata
- Storage manager → Catalog service: uses catalog (evidence: `pdb/src/storageManager/source/PDBStorageManagerFrontend.cc`)
- Storage manager → Buffer manager: manages pages
- Application user → GMM workload: runs example
- GMM workload → PDB client: submits computations (evidence: `applications/TestGMM/TestGmmLazy.cc`)
- GMM workload → Lambda expressions: builds expressions (evidence: `applications/TestGMM/TestGmmLazy.cc`)
- GMM model → Object model: uses object types (evidence: `applications/TestGMM/sharedLibraries/headers/GmmModel.h`)
- Application user → LDA workload: runs example
- LDA workload → PDB client: uses client (evidence: `applications/TestLDA/TestLDA.cc`)
- LDA topic sampling → Object model: creates assignments
- Application user → Linear algebra: uses library
- Application user → Matrix multiplication: runs example
