# pahulmeet/sansa-python-wrapper architecture

> Architecture diagram of the GitHub repository pahulmeet/sansa-python-wrapper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pahulmeet/sansa-python-wrapper
- Repository: https://github.com/pahulmeet/sansa-python-wrapper

## Overview

##### Main Components
- User Interaction Layer: This includes the Jupyter Notebook environment (as found in the ml_notebook, rdf, and query example directories) where the user executes notebooks and directly calls the pysansa library.
- pysansa Package: This is the core of the project and is subdivided into several layers:
- RDF Layer (pysansa/rdf): Responsible for reading, parsing, and processing RDF data.
- Query Layer (pysansa/query): Provides methods to perform SPARQL queries on RDF datasets.
- ML Layer (pysansa/ml): Contains functionality for machine learning operations on the semantic data.
- myjars: This indicates the inclusion of external JAR files (Java dependencies) that interface with the SANSA-Stack.
- Data Sources: The ‘data’ folder holds RDF data files such as rdf.nt, alternative_bigger_rdf.nt, and movie.nt.
- External Dependencies/Infrastructure:
- Apache Spark: Used for distributed processing. The README details the Spark installation along with starting a Spark master and worker. This component should be shown as a cluster component.
- Hadoop: Configured as the underlying distributed file system and processing engine.
- Java / Maven / Scala: These underpin the Java-based SANSA-Stack libraries. Although they are part of the installation process, they are also part of the external tech stack that the pysansa jobs rely upon.
- SANSA JAR Library: Downloaded externally and used by the pysansa modules (especially the RDF layer) via spark-submit.

##### Relationships and Interactions
- The user (via a Jupyter notebook) sends commands, invoking methods from the pysansa package.
- The pysansa package, through its distinct layers, abstracts the complexities of parsing RDF data (RDF Layer), executing queries (Query Layer), and performing machine learning tasks (ML Layer).
- The underlying Spark cluster processes the requests initiated by pysansa (for example, running distributed SPARQL queries or ML algorithms).
- Hadoop acts as the file system for storing large RDF datasets and facilitating data exchange.
- External JAR libraries (from the ‘myjars’ directory) are called by the Python wrapper when interacting with the Java-based SANSA components.

##### Architectural Patterns and Design Principles
- Layered Architecture: The code is segmented into distinct logical layers corresponding to data ingestion (RDF), querying (Query), and data science (ML) operations.
- Wrapper/Facade Pattern: pysansa functions as a façade that simplifies the interaction with complex underlying systems (i.e., SANSA-Stack, Spark, and Hadoop).
- Separation of Concerns: The clear separation of notebooks, pysansa modules, and external configuration files indicates that the system design emphasizes modularity.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction Layer
    subgraph "User Interaction Layer"
        UI_ML["Jupyter Notebook (ML)"]:::ui
        UI_RDF["Jupyter Notebook (RDF)"]:::ui
        UI_Query["Jupyter Notebook (Query)"]:::ui
    end

    %% pysansa Python Wrapper with Internal Layers
    subgraph "pysansa Python Wrapper"
        RDF_Layer["RDF Layer"]:::python
        Query_Layer["Query Layer"]:::python
        ML_Layer["ML Layer"]:::python
        myjars["myjars (SANSA JARs)"]:::python
    end

    %% Data Sources
    Data_Sources["Data Sources"]:::data

    %% External Infrastructure Components
    subgraph "External Infrastructure"
        subgraph "Apache Spark Cluster"
            Spark_Master["Spark Master"]:::external
            Spark_Worker["Spark Worker"]:::external
        end
        Hadoop["Hadoop Infrastructure"]:::external
        JavaMavenScala["Java/Maven/Scala"]:::external
    end

    %% Relationships: User Interface to pysansa Layers
    UI_ML -->|"calls"| ML_Layer
    UI_RDF -->|"calls"| RDF_Layer
    UI_Query -->|"calls"| Query_Layer

    %% Relationships: pysansa Layers and Data/External
    RDF_Layer -->|"loads RDF"| Data_Sources
    RDF_Layer -->|"invokes"| myjars

    Query_Layer -->|"executes SPARQL"| Spark_Master
    ML_Layer -->|"triggers ML jobs"| Spark_Master

    %% Spark Cluster Internal Communication
    Spark_Master -->|"coordinates with"| Spark_Worker
    Spark_Master -->|"reads/writes"| Hadoop
    Spark_Worker -->|"reads/writes"| Hadoop

    %% External Dependencies
    RDF_Layer -->|"depends on"| JavaMavenScala
    Query_Layer -->|"depends on"| JavaMavenScala
    ML_Layer -->|"depends on"| JavaMavenScala

    %% Optional Interaction among Notebooks
    UI_ML ---|"interactive"| UI_RDF
    UI_RDF ---|"interactive"| UI_Query

%% Click Events
click UI_ML "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/ml_notebook/ML_Notebook.ipynb"
click UI_RDF "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/rdf/rdfExampleNotebook.ipynb"
click UI_Query "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/query/queryExampleNotebook.ipynb"

click RDF_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/rdf/README.md"
click RDF_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/rdf/rdf.py"
click RDF_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/rdf/rdf_read.py"

click Query_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/query/README.md"
click Query_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/query/query.py"

click ML_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/ml/README.md"
click ML_Layer "https://github.com/pahulmeet/sansa-python-wrapper/blob/main/pysansa/ml/ml.py"

click myjars "https://github.com/pahulmeet/sansa-python-wrapper/tree/main/pysansa/myjars/"

click Data_Sources "https://github.com/pahulmeet/sansa-python-wrapper/tree/main/data/"

%% Style Definitions
classDef ui fill:#f9c,stroke:#333,stroke-width:2px;
classDef python fill:#bbf,stroke:#333,stroke-width:2px;
classDef data fill:#bfb,stroke:#333,stroke-width:2px;
classDef external fill:#ffc,stroke:#333,stroke-width:2px;
```
