How dimitrijejankov/plinycompute works
PlinyCompute is a C++ platform for building and running distributed data and computation workloads in standalone or distributed clusters. A user submits computations and manages datasets through client APIs; computation planning and execution run on the platform, while storage and catalog services manage the data and its metadata. The tree also contains representative machine-learning and linear-algebra workloads. The README documents broad capabilities, but the excerpts do not expose every server-side dispatch path, so some documented workflow links are shown without claiming implementation details.
Main entry points: PDBClient.cc PDB client · Pipeline.cc Execution pipeline · TestGmmLazy.cc GMM workload · PDBCatalogClient.h Catalog client
Components 23
Client and query APIs
- PDB client
pdb/src/client/source/PDBClient.cc - Catalog client
pdb/src/client/headers/PDBCatalogClient.h - Storage client
pdb/src/client/headers/PDBDistributedStorageClient.h - Query analyzer
pdb/src/client/source/QueryGraphAnalyzer.cc - Lambda expressions
pdb/src/lambdas/headers/Lambda.h - Computation client
pdb/src/client/source/PDBComputationClient.cc
Computation runtime
- Computation server
pdb/src/computationServer/sources/PDBComputationServerFrontend.cc - Physical optimizer
pdb/src/computationServer/sources/physicalOptimizer/PDBAbstractPhysicalNode.cc - Execution pipeline
pdb/src/pipeline/source/pipeline/Pipeline.cc - Join pipeline
pdb/src/pipeline/source/pipeline/JoinBroadcastPipeline.cc - Object model
pdb/src/objectModel/headers/Handle.h
Data and metadata
- Catalog service
pdb/src/catalog/source/CatalogServer.cc - Catalog metadata
pdb/src/catalog/headers/PDBCatalogStorage.h - Distributed storage
pdb/src/distributedStorage/source/PDBDistributedStorage.cc - Storage manager
pdb/src/storageManager/source/PDBStorageManagerFrontend.cc - Buffer manager
pdb/src/bufferManager/source/PDBBufferManagerImpl.cc
Example workloads
- GMM workload
applications/TestGMM/TestGmmLazy.cc - GMM model
applications/TestGMM/sharedLibraries/headers/GmmModel.h - LDA workload
applications/TestLDA/TestLDA.cc - LDA topic sampling
applications/TestLDA/sharedLibraries/headers/LDADocWordTopicJoin.h - Linear algebra
applications/TestLinearAlgebra/linearAlgebraDSL/source/LAParserHelperFunctions.cc - Matrix multiplication
applications/TestMatrixMultiply/TestMatrixMultiply.cc
Other components
- Application user
Connections 24
- Application user → PDB client: submits work
- PDB client → Catalog client: manages metadata
- PDB client → Storage client: stores data
- PDB client → Computation client: runs computations
- PDB client → Query analyzer: analyzes queries
- Computation client → Computation server: submits computation
- Computation server → Physical optimizer: plans execution
- Physical optimizer → Execution pipeline: builds execution
- Execution pipeline → Join pipeline: uses join pipeline
- Execution pipeline → Object model: processes objects
- Storage client → Catalog client: uses catalog
pdb/src/client/headers/PDBDistributedStorageClient.h - Distributed storage → Catalog client: uses catalog
pdb/src/distributedStorage/source/PDBDistributedStorage.cc - Catalog service → Catalog metadata: manages metadata
- Storage manager → Catalog service: uses catalog
pdb/src/storageManager/source/PDBStorageManagerFrontend.cc - Storage manager → Buffer manager: manages pages
- Application user → GMM workload: runs example
- GMM workload → PDB client: submits computations
applications/TestGMM/TestGmmLazy.cc - GMM workload → Lambda expressions: builds expressions
applications/TestGMM/TestGmmLazy.cc - GMM model → Object model: uses object types
applications/TestGMM/sharedLibraries/headers/GmmModel.h - Application user → LDA workload: runs example
- LDA workload → PDB client: uses client
applications/TestLDA/TestLDA.cc - LDA topic sampling → Object model: creates assignments
- Application user → Linear algebra: uses library
- Application user → Matrix multiplication: runs example