# hrefto/shorterdb architecture

> Architecture diagram of the GitHub repository hrefto/shorterdb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hrefto/shorterdb
- Repository: https://github.com/hrefto/shorterdb

## Overview

##### Project type and purpose
- From the file tree and absence of a filled-out README, we see:
- This is a standalone key-value store server implementation in Rust.
- It uses a Protocol Buffers schema (proto/commands.proto) for its network API.
- The “kv” folder contains the core storage engine (Memtable, WAL, SSTable).
- server.rs and main.rs wire up a networked RPC server around the engine.
- randomdata.py is a test/data generator.

##### File structure
- Top-level artifacts:
- Cargo.toml / build.rs → Rust build and dependency management.
- proto/commands.proto → gRPC or protobuf definitions.
- randomdata.py → external data-generation script.
- src/:
- main.rs → application entrypoint (initializes config, logging, server).
- server.rs → RPC server setup (imports the generated protobuf bindings and exposes handlers).
- errors.rs → centralized error types.
- kv/ (core storage engine):
– db.rs → high-level DB API (get, put, delete, iterate).
– wal.rs → write-ahead-log implementation for durability.
– memtable.rs → in-memory memtable for writes before flush.
– sst.rs → SSTable file format and reader logic.
– utils.rs → helper functions (file I/O, encoding/decoding).

##### README
- It’s still “todo!()”, so rely on file names and module responsibilities to infer architecture.

##### Detailed capture of architectural elements
- Lifecycle of a write and read.
- How the protobuf definitions flow into generated code (via build.rs) and into server.rs.

## Diagram (Mermaid)

```mermaid
flowchart TD
    Client["gRPC Client"]:::external

    subgraph "Network / API"
        Proto["ProtoBuf Schema"]:::api
        BuildScript["Build Script"]:::build
        Generated["Generated Bindings"]:::api
    end

    Client -->|"gRPC Request/Response"| Proto
    Proto -->|"compile proto"| BuildScript
    BuildScript -->|"generates code"| Generated
    Generated -->|"invokes"| Server

    Main["Application Entry (main.rs)"]:::api
    Server["RPC Server (server.rs)"]:::api
    Errors["Error Definitions (errors.rs)"]:::engine

    Main --> Server
    Main --> Errors

    subgraph "KV Engine"
        DBAPI["DB API (db.rs)"]:::engine
        WAL["Write-Ahead Log (wal.rs)"]:::disk
        Memtable["Memtable (memtable.rs)"]:::memory
        SST["SSTable (sst.rs)"]:::disk
        Utils["Utils (utils.rs)"]:::engine
    end

    Server -->|"handles RPC calls"| DBAPI
    DBAPI -->|"get/iterate"| Memtable
    DBAPI -->|"put/delete"| WAL
    DBAPI --> Utils

    Memtable -->|"flush when full"| SST
    Memtable -->|"miss lookup"| SST

    subgraph "Storage"
        Disk["Filesystem"]:::disk
    end

    WAL -->|"append entries"| Disk
    SST -->|"store/read files"| Disk

    subgraph "Tooling / Build"
        Cargo["Cargo.toml"]:::build
        Random["Test Data Generator (randomdata.py)"]:::build
    end

    Random -->|"sends test requests"| Server
    Cargo --> BuildScript

    click Proto "https://github.com/hrefto/shorterdb/blob/master/proto/commands.proto"
    click BuildScript "https://github.com/hrefto/shorterdb/blob/master/build.rs"
    click Main "https://github.com/hrefto/shorterdb/blob/master/src/main.rs"
    click Server "https://github.com/hrefto/shorterdb/blob/master/src/server.rs"
    click Errors "https://github.com/hrefto/shorterdb/blob/master/src/errors.rs"
    click DBAPI "https://github.com/hrefto/shorterdb/blob/master/src/kv/db.rs"
    click WAL "https://github.com/hrefto/shorterdb/blob/master/src/kv/wal.rs"
    click Memtable "https://github.com/hrefto/shorterdb/blob/master/src/kv/memtable.rs"
    click SST "https://github.com/hrefto/shorterdb/blob/master/src/kv/sst.rs"
    click Utils "https://github.com/hrefto/shorterdb/blob/master/src/kv/utils.rs"
    click Random "https://github.com/hrefto/shorterdb/blob/master/randomdata.py"
    click Cargo "https://github.com/hrefto/shorterdb/blob/master/Cargo.toml"

    classDef api fill:#BBDEFB,stroke:#2196F3
    classDef memory fill:#C8E6C9,stroke:#388E3C
    classDef disk fill:#FFE0B2,stroke:#F57C00
    classDef build fill:#E0E0E0,stroke:#9E9E9E
    classDef engine fill:#F5F5F5,stroke:#9E9E9E
    classDef external fill:#D1C4E9,stroke:#7E57C2
```
