# kuberlab/pluk architecture

> Architecture diagram of the GitHub repository kuberlab/pluk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kuberlab/pluk
- Repository: https://github.com/kuberlab/pluk

## Overview

##### Main Components
- Pluk Server: This is the core backend process (pluksrv.go) responsible for handling HTTP requests, managing data chunks, file metadata, and communicating with the DB.
- API Layer: Under pkg/api, you have RESTful API endpoints (with functions for handling datasets, files, and versioning) that connect front-end CLI clients and possibly internal FUSE connections.
- Database: Under pkg/db and pkg/db/gorm, all interactions with the chosen database (sqlite3 by default, but also mysql/postgres are supported) are defined. This database stores the virtual filesystem’s metadata, dataset versions, and file chunks info.
- FUSE Filesystem: The plukefs binary (and its supporting code in pkg/fuse) allows mounting datasets as read-only filesystems. It provides an interface between the kernel’s filesystem calls and the underlying dataset management logic.
- CLI Tool – kdataset: Located in the cmd/kdataset folder, this tool offers a command-line interface for operations such as push, pull, list, version management, and deletion.
- Auxiliary Tools: Additional commands such as delete_wrong_chunks and update_chunks handle specific maintenance or synchronization tasks.
- gRPC Communication: The pkg/grpc package indicates that there is support for gRPC-based communication, perhaps for internal communication (such as master-slave sync where masters are defined via the environment variable).
- External Configuration & Environment: Environment variables (like DATA_DIR, DB_NAME, AUTH_VALIDATION, and MASTERS) play a crucial role in configuring the system and its interactions with external services (e.g., an authentication proxy).

##### Relationships and Interactions
- The Pluk Server (backend) exposes a REST API (and possibly WebSocket endpoints) to handle dataset operations.
- Requests may come from both external clients (through the kdataset CLI) and internally from the FUSE filesystem (plukefs) when mounting a dataset.
- The server communicates with the Database layer to read/write dataset metadata, chunk data references, version info, etc.
- gRPC communications are utilized for internal master-slave replication/communication where slave instances interact with a master (using environment-specified internal keys).
- The CLI tool (kdataset) interacts with the Pluk Server via HTTP (or through a configuration provided pluk_url) to perform its operations.
- Docker containers (referenced by Dockerfile and Dockerfile.plukefs) encapsulate the necessary environment, ensuring correct volumes for DATA_DIR and DB access.
- Maintenance utilities (delete_wrong_chunks, update_chunks) likely have shorter, specialized data flows targeting cleanup or synchronization tasks.

##### Important Architectural Patterns and Design Principles
- Modular design: Different commands and functionalities are separated into distinct binaries (pluk, plukefs, kdataset) and packages (pkg/api, pkg/db, pkg/fuse, etc.), reflecting a separation of concerns.
- Configurability and environment-based behavior: Use of environment variables to configure the storage (DATA_DIR), DB type/parameters, and authentication tools.
- Client–server model: With a clearly defined API layer that decouples client tools (CLI, FUSE) from the core data management logic.
- Containerization: Support and guidance for running the system inside Docker, ensuring portability.
- Extensibility: Through clear command-line interfaces and configuration files (YAML for kdataset), it is designed to be integrated and extended as needed.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients Subgraph
    subgraph "Clients"
        CLI["CLI Tool: kdataset"]:::client
        FUSE["FUSE Filesystem (plukefs)"]:::client
        DWCH["delete_wrong_chunks"]:::client
        UCH["update_chunks"]:::client
    end

    %% Backend Services Subgraph
    subgraph "Backend Services"
        SRV["Pluk HTTP Server"]:::server
        API["API Layer"]:::api
        DB["Database (SQLite/MySQL/Postgres)"]:::db
        GRPC["gRPC Communication"]:::grpc
        CONF["External Config & Environment"]:::config
    end

    %% Connections from Clients to Backend Services
    CLI -->|"REST_requests"| SRV
    FUSE -->|"Mount_requests"| SRV
    DWCH -->|"Maintenance"| SRV
    UCH -->|"Maintenance"| SRV

    %% Internal Backend Connections
    SRV -->|"API_calls"| API
    SRV -->|"DB_operations"| DB
    SRV -->|"gRPC_sync"| GRPC
    SRV -->|"Configures"| CONF

    %% Click Events
    click SRV "https://github.com/kuberlab/pluk/blob/master/pluksrv.go"
    click API "https://github.com/kuberlab/pluk/tree/master/pkg/api"
    click DB "https://github.com/kuberlab/pluk/tree/master/pkg/db"
    click DB "https://github.com/kuberlab/pluk/tree/master/pkg/db/gorm"
    click FUSE "https://github.com/kuberlab/pluk/tree/master/cmd/plukefs"
    click FUSE "https://github.com/kuberlab/pluk/tree/master/pkg/fuse"
    click CLI "https://github.com/kuberlab/pluk/tree/master/cmd/kdataset"
    click DWCH "https://github.com/kuberlab/pluk/tree/master/cmd/delete_wrong_chunks"
    click UCH "https://github.com/kuberlab/pluk/tree/master/cmd/update_chunks"
    click GRPC "https://github.com/kuberlab/pluk/tree/master/pkg/grpc"
    click CONF "https://github.com/kuberlab/pluk/tree/master/pkg/config"

    %% Styles
    classDef client fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef server fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef api fill:#A9DFBF,stroke:#145A32,stroke-width:2px;
    classDef db fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef grpc fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef config fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
```
