# mystenlabs/walrus architecture

> Architecture diagram of the GitHub repository mystenlabs/walrus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mystenlabs/walrus
- Repository: https://github.com/mystenlabs/walrus

## Overview

##### Overall project type and purpose
- Walrus is a decentralized blob store that leverages the Sui blockchain for coordination and governance. Its core functionality revolves around storing and transferring “blobs” (likely large binary data) in a distributed, secure manner.
- The project uses smart contracts (written in Move under the contracts and testnet-contracts directories) for on-chain coordination and governance, and it is written largely in Rust (organized in multiple “crates”) for off-chain processing, client interactions, and service operations.

##### Break the system into its principal components
a. Blockchain / Smart Contracts Layer:
- Smart contracts that govern and coordinate operations. These are stored under directories like “contracts” and “testnet-contracts”.
- That these contracts (written in Move) are deployed on the Sui network to manage blob storage, governance protocols, and other critical on-chain data.

b. Core Library & SDK:
- “walrus-core”: This crate contains the core types and encoding/authentication mechanisms.
- “walrus-sdk”: The SDK for interacting with storage nodes. It abstracts communication and client operations.

c. Service / Node Functionality:
- “walrus-service”: The main crate that provides both client and server functionality, containing binaries such as “walrus” and “walrus-node”. It handles storage backend operations, node management, and API endpoints.
- “walrus-sui”: Handles interactions with Sui smart contracts and other related types.
- “walrus-utils”: Contains supporting utility functions.

d. Ancillary Components & Tools:
- Other crates such as “checkpoint-downloader”, “walrus-orchestrator”, “walrus-proxy”, “walrus-simtest”, “walrus-stress”, and “walrus-test-utils” provide deployment orchestration, performance stress testing, simulation testing, proxying for metrics, and other auxiliary functions.

e. Deployment and Automation:
- “docker” and “setup” directories provide containerized deployments and configuration. They include Dockerfiles and docker-compose files, which are crucial for running the system in production or testing environments.
- “scripts” provide tools for local testing, generating release notes, and testbed setups.

##### Interactions and relationships
- Walrus-service nodes interacting with the blockchain layer (smart contracts on Sui) via the walrus-sui component.
- That the core libraries (walrus-core and walrus-utils) are utility blocks used by other Rust crates.
- For deployment, add a separate block for containerization and orchestration (Docker setups) which encapsulate several of these components for production and testbed environments.

##### Architectural patterns and design principles to emphasize
- Modularity: Each crate has clearly defined responsibilities (e.g., core encoding, storage node management, blockchain interactions).
- Separation of concerns: Smart contracts (on-chain governance) are isolated from off-chain processing (the Rust-based services) to allow independent scaling and maintenance.
- Distributed design: The system should highlight decoupled client interactions (via the SDK) and a decentralized service layer with nodes, stress tools, and orchestrators.
- Microservices-style deployment: Even if all components are built in a single repository, the Docker and orchestration files imply that the service, proxy, orchestrator, and stress tools are deployed as separate microservices.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Blockchain / Smart Contracts Layer"
        A["On-chain Move Smart Contracts"]:::onchain
        B["Testnet Smart Contracts"]:::onchain
    end

    subgraph "Core Libraries & SDK"
        C["Walrus Core"]:::core
        D["Walrus SDK"]:::sdk
    end

    subgraph "Service / Node Functionality"
        E["Walrus Service"]:::service
        F["Walrus Sui"]:::service
        G["Walrus Utils"]:::core
    end

    subgraph "Ancillary Components & Tools"
        H["Walrus Orchestrator"]:::ancillary
        I["Walrus Proxy"]:::ancillary
        J["Walrus Simtest"]:::ancillary
        K["Walrus Stress"]:::ancillary
        L["Walrus Test Utils"]:::ancillary
        M["Checkpoint Downloader"]:::ancillary
    end

    subgraph "Deployment & Automation"
        N["Docker Deployments"]:::deployment
        O["Setup & Scripts"]:::deployment
    end

    %% Connections
    D -->|"API_Call"| E
    E -->|"Blockchain_Query"| F
    F -->|"On_Chain_Call"| A
    F -->|"On_Chain_Call"| B
    E -->|"Uses"| C
    E -->|"Uses"| G
    D -->|"Uses"| C

    H -->|"Orchestration"| E
    I -->|"Metrics_Aggregation"| E
    J -->|"Simulation_Test"| E
    K -->|"Stress_Test"| E
    L -->|"Test_Utilities"| E
    M -->|"Data_Backup"| E

    N -->|"Container_Run"| E
    O -->|"Setup_Config"| E

    %% Click Events
    click A "https://github.com/mystenlabs/walrus/tree/main/contracts"
    click B "https://github.com/mystenlabs/walrus/tree/main/testnet-contracts"
    click C "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-core"
    click D "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-sdk"
    click E "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-service"
    click F "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-sui"
    click G "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-utils"
    click H "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-orchestrator"
    click I "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-proxy"
    click J "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-simtest"
    click K "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-stress"
    click L "https://github.com/mystenlabs/walrus/tree/main/crates/walrus-test-utils"
    click M "https://github.com/mystenlabs/walrus/tree/main/crates/checkpoint-downloader"
    click N "https://github.com/mystenlabs/walrus/tree/main/docker"
    click O "https://github.com/mystenlabs/walrus/tree/main/setup"

    %% Styles
    classDef onchain fill:#add8e6,stroke:#0000ff,stroke-width:2px;
    classDef core fill:#ccffcc,stroke:#008000,stroke-width:2px;
    classDef sdk fill:#ffffcc,stroke:#ffd700,stroke-width:2px;
    classDef service fill:#ffe4b5,stroke:#ff8c00,stroke-width:2px;
    classDef ancillary fill:#e6e6fa,stroke:#800080,stroke-width:2px;
    classDef deployment fill:#f4a460,stroke:#d2691e,stroke-width:2px;
```
