# astrolabsoftware/spark3d architecture

> Architecture diagram of the GitHub repository astrolabsoftware/spark3d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/astrolabsoftware/spark3d
- Repository: https://github.com/astrolabsoftware/spark3d

## Overview

spark3D extends Spark SQL/RDD processing for three-dimensional datasets, exposing Scala and Python APIs. The main workflow accepts spatial data, represents it with 3D shapes, partitions or repartitions it, then supports spatial queries and visualization. The graph preserves both API surfaces and the documented visualization capability. Edges follow SOURCE INDEX references or the README; unsampled implementation details limit finer-grained links, especially for visualization internals.

Main entry points: [`pyspark3d/__init__.py`](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/__init__.py) (Python API), [`src/main/scala/com/spark3d/Queries.scala`](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Queries.scala) (Spatial Query API), [`src/main/scala/com/spark3d/Partitioners.scala`](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Partitioners.scala) (Partition Selection), [`src/main/scala/com/spark3d/geometryObjects/Shape3D.scala`](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/Shape3D.scala) (Shape Operations)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_apis["User APIs"]
  node_scala_api["Scala API<br/>[package.scala]"]
  node_python_api["Python API<br/>[__init__.py]"]
  node_py_config["Python Configuration<br/>[pyspark3d_conf.py]"]
  node_py_version["Package Version<br/>[version.py]"]
  node_py_convert["Data Conversion<br/>[converters.py]"]
end

subgraph group_spatial["Spatial Model"]
  node_py_geometry["Python Geometry<br/>[geometryObjects.py]"]
  node_point["3D Points<br/>[Point3D.scala]"]
  node_shape["Shape Operations<br/>[Shape3D.scala]"]
  node_box["Box Envelopes<br/>[BoxEnvelope.scala]"]
  node_shell["Shell Envelopes"]
  node_coord_utils["Coordinate Utilities<br/>[Utils.scala]"]
end

subgraph group_partition["Partitioning"]
  node_repartition["Data Repartitioning"]
  node_partition_factory["Partition Selection<br/>[Partitioners.scala]"]
  node_spatial_partitioner["Partitioner Base"]
  node_octree["Octree Grid<br/>[Octree.scala]"]
  node_octree_partitioner["Octree Partitioner"]
  node_onion_partitioner["Onion Partitioner"]
  node_onion_grid["Onion Grid"]
  node_key_partitioner["Key Partitioner"]
end

subgraph group_analysis["Queries And Display"]
  node_query_api["Spatial Query API<br/>[Queries.scala]"]
  node_window_query["Window Query<br/>[WindowQuery.scala]"]
  node_knn["Nearest Neighbors<br/>[KNN.scala]"]
  node_visualisation["Data Visualization<br/>[visualisation.py]"]
end

node_spark_user(("Spark User"))
node_spark["Apache Spark"]

node_spark_user -->|"uses Scala API"| node_scala_api
node_spark_user -->|"uses Python API"| node_python_api
node_scala_api -->|"exposes repartitioning"| node_repartition
node_python_api -->|"loads configuration"| node_py_config
node_py_config -->|"reads version"| node_py_version
node_python_api -->|"uses converters"| node_py_convert
node_python_api -.->|"exposes geometry"| node_py_geometry
node_python_api -->|"bridges through Py4J"| node_spark
node_scala_api -->|"extends Spark"| node_spark
node_point -->|"implements shape"| node_shape
node_shape -->|"uses envelopes"| node_box
node_shape -->|"uses conversions"| node_coord_utils
node_box -->|"uses points"| node_point
node_shell -->|"uses points"| node_point
node_repartition -->|"selects partitioner"| node_partition_factory
node_repartition -->|"uses key partitioner"| node_key_partitioner
node_partition_factory -->|"uses partitioner base"| node_spatial_partitioner
node_partition_factory -->|"selects octree"| node_octree_partitioner
node_partition_factory -->|"selects onion"| node_onion_partitioner
node_octree_partitioner -->|"uses grid"| node_octree
node_onion_partitioner -->|"uses grid"| node_onion_grid
node_octree -->|"indexes envelopes"| node_box
node_onion_grid -->|"uses shells"| node_shell
node_query_api -->|"delegates window query"| node_window_query
node_window_query -->|"tests intersections"| node_shape
node_query_api -->|"uses points"| node_point
node_query_api -->|"uses shells"| node_shell
node_query_api -->|"uses boxes"| node_box
node_spark_user -->|"visualizes datasets"| node_visualisation

click node_scala_api "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/package.scala"
click node_python_api "https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/__init__.py"
click node_py_config "https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/pyspark3d_conf.py"
click node_py_version "https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/version.py"
click node_py_convert "https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/converters.py"
click node_py_geometry "https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/geometryObjects.py"
click node_point "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/Point3D.scala"
click node_shape "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/Shape3D.scala"
click node_box "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/BoxEnvelope.scala"
click node_shell "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/ShellEnvelope.scala"
click node_coord_utils "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/utils/Utils.scala"
click node_repartition "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Repartitioning.scala"
click node_partition_factory "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Partitioners.scala"
click node_spatial_partitioner "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/SpatialPartitioner.scala"
click node_octree "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/Octree.scala"
click node_octree_partitioner "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/OctreePartitioner.scala"
click node_onion_partitioner "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioner.scala"
click node_onion_grid "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioning.scala"
click node_key_partitioner "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/KeyPartitioner.scala"
click node_query_api "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Queries.scala"
click node_window_query "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialOperator/WindowQuery.scala"
click node_knn "https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialOperator/KNN.scala"
click node_visualisation "https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/visualisation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scala_api,node_python_api,node_py_config,node_py_version,node_py_convert,node_spark_user toneBlue
class node_py_geometry,node_point,node_shape,node_box,node_shell,node_coord_utils toneAmber
class node_repartition,node_partition_factory,node_spatial_partitioner,node_octree,node_octree_partitioner,node_onion_partitioner,node_onion_grid,node_key_partitioner toneMint
class node_query_api,node_window_query,node_knn,node_visualisation toneRose
class node_spark toneIndigo
```

## Components

### User APIs

- [Scala API](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/package.scala): `src/main/scala/com/spark3d/package.scala`
- [Python API](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/__init__.py): `pyspark3d/__init__.py`
- [Python Configuration](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/pyspark3d_conf.py): `pyspark3d/pyspark3d_conf.py`
- [Package Version](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/version.py): `pyspark3d/version.py`
- [Data Conversion](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/converters.py): `pyspark3d/converters.py`

### Spatial Model

- [Python Geometry](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/geometryObjects.py): `pyspark3d/geometryObjects.py`
- [3D Points](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/Point3D.scala): `src/main/scala/com/spark3d/geometryObjects/Point3D.scala`
- [Shape Operations](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/Shape3D.scala): `src/main/scala/com/spark3d/geometryObjects/Shape3D.scala`
- [Box Envelopes](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/BoxEnvelope.scala): `src/main/scala/com/spark3d/geometryObjects/BoxEnvelope.scala`
- [Shell Envelopes](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/geometryObjects/ShellEnvelope.scala): `src/main/scala/com/spark3d/geometryObjects/ShellEnvelope.scala`
- [Coordinate Utilities](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/utils/Utils.scala): `src/main/scala/com/spark3d/utils/Utils.scala`

### Partitioning

- [Data Repartitioning](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Repartitioning.scala): `src/main/scala/com/spark3d/Repartitioning.scala`
- [Partition Selection](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Partitioners.scala): `src/main/scala/com/spark3d/Partitioners.scala`
- [Partitioner Base](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/SpatialPartitioner.scala): `src/main/scala/com/spark3d/spatialPartitioning/SpatialPartitioner.scala`
- [Octree Grid](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/Octree.scala): `src/main/scala/com/spark3d/spatialPartitioning/Octree.scala`
- [Octree Partitioner](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/OctreePartitioner.scala): `src/main/scala/com/spark3d/spatialPartitioning/OctreePartitioner.scala`
- [Onion Partitioner](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioner.scala): `src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioner.scala`
- [Onion Grid](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioning.scala): `src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioning.scala`
- [Key Partitioner](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialPartitioning/KeyPartitioner.scala): `src/main/scala/com/spark3d/spatialPartitioning/KeyPartitioner.scala`

### Queries And Display

- [Spatial Query API](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/Queries.scala): `src/main/scala/com/spark3d/Queries.scala`
- [Window Query](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialOperator/WindowQuery.scala): `src/main/scala/com/spark3d/spatialOperator/WindowQuery.scala`
- [Nearest Neighbors](https://github.com/astrolabsoftware/spark3d/blob/master/src/main/scala/com/spark3d/spatialOperator/KNN.scala): `src/main/scala/com/spark3d/spatialOperator/KNN.scala`
- [Data Visualization](https://github.com/astrolabsoftware/spark3d/blob/master/pyspark3d/visualisation.py): `pyspark3d/visualisation.py`

### Other components

- **Spark User**
- **Apache Spark**

## Connections

- Spark User → Scala API: uses Scala API (evidence: `README.md`)
- Spark User → Python API: uses Python API (evidence: `README.md`)
- Scala API → Data Repartitioning: exposes repartitioning (evidence: `src/main/scala/com/spark3d/package.scala`)
- Python API → Python Configuration: loads configuration (evidence: `pyspark3d/__init__.py`)
- Python Configuration → Package Version: reads version (evidence: `pyspark3d/pyspark3d_conf.py`)
- Python API → Data Conversion: uses converters (evidence: `pyspark3d/converters.py`)
- Python API → Python Geometry: exposes geometry (evidence: `pyspark3d/geometryObjects.py`)
- Python API → Apache Spark: bridges through Py4J (evidence: `README.md`)
- Scala API → Apache Spark: extends Spark (evidence: `README.md`)
- 3D Points → Shape Operations: implements shape (evidence: `src/main/scala/com/spark3d/geometryObjects/Point3D.scala`)
- Shape Operations → Box Envelopes: uses envelopes (evidence: `src/main/scala/com/spark3d/geometryObjects/Shape3D.scala`)
- Shape Operations → Coordinate Utilities: uses conversions (evidence: `src/main/scala/com/spark3d/geometryObjects/Shape3D.scala`)
- Box Envelopes → 3D Points: uses points (evidence: `src/main/scala/com/spark3d/geometryObjects/BoxEnvelope.scala`)
- Shell Envelopes → 3D Points: uses points (evidence: `src/main/scala/com/spark3d/geometryObjects/ShellEnvelope.scala`)
- Data Repartitioning → Partition Selection: selects partitioner (evidence: `src/main/scala/com/spark3d/Repartitioning.scala`)
- Data Repartitioning → Key Partitioner: uses key partitioner (evidence: `src/main/scala/com/spark3d/Repartitioning.scala`)
- Partition Selection → Partitioner Base: uses partitioner base (evidence: `src/main/scala/com/spark3d/Partitioners.scala`)
- Partition Selection → Octree Partitioner: selects octree (evidence: `src/main/scala/com/spark3d/Partitioners.scala`)
- Partition Selection → Onion Partitioner: selects onion (evidence: `src/main/scala/com/spark3d/Partitioners.scala`)
- Octree Partitioner → Octree Grid: uses grid (evidence: `src/main/scala/com/spark3d/spatialPartitioning/OctreePartitioner.scala`)
- Onion Partitioner → Onion Grid: uses grid (evidence: `src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioner.scala`)
- Octree Grid → Box Envelopes: indexes envelopes (evidence: `src/main/scala/com/spark3d/spatialPartitioning/Octree.scala`)
- Onion Grid → Shell Envelopes: uses shells (evidence: `src/main/scala/com/spark3d/spatialPartitioning/OnionPartitioning.scala`)
- Spatial Query API → Window Query: delegates window query (evidence: `src/main/scala/com/spark3d/Queries.scala`)
- Window Query → Shape Operations: tests intersections (evidence: `src/main/scala/com/spark3d/spatialOperator/WindowQuery.scala`)
- Spatial Query API → 3D Points: uses points (evidence: `src/main/scala/com/spark3d/Queries.scala`)
- Spatial Query API → Shell Envelopes: uses shells (evidence: `src/main/scala/com/spark3d/Queries.scala`)
- Spatial Query API → Box Envelopes: uses boxes (evidence: `src/main/scala/com/spark3d/Queries.scala`)
- Spark User → Data Visualization: visualizes datasets (evidence: `README.md`)
