# dicedb/dice architecture

> Architecture diagram of the GitHub repository dicedb/dice, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dicedb/dice
- Repository: https://github.com/dicedb/dice

## Overview

##### Project type and purpose
- DiceDB is an open-source, reactive, in-memory database server written in Go.
- It provides a Redis-like TCP protocol on port 7379, supports real-time subscriptions to data changes, and persists data via AOF/WAL. It also ships with a static documentation site built in Astro.

##### File structure
- Top‐level “cmd/” and “main.go” – entrypoints for the server binary.
- “internal/server/ironhawk/” – network layer using epoll/kqueue (the Reactor pattern).
- “internal/iomultiplexer/” – cross-platform I/O multiplexing (epoll/kqueue abstractions).
- “internal/cmd/” – individual command implementations (GET, SET, ZADD, etc.).
- “internal/eval/” – evaluation engine (parsing, execution, intermediate data types).
- “internal/store/” – in-memory data structures, eviction, expiration.
- “internal/wal/” and “internal/store/aof.go” – persistence via write‐ahead log and append-only file.
- “internal/shard*” packages – sharding infrastructure for horizontal scaling within a single process.
- “internal/watchmanager/” – observer pattern for reactive query subscriptions.
- “internal/observability/” and “internal/logger/” – metrics, telemetry, logging.
- “docs/” – static site source (Astro framework) for documentation.
- “examples/” – sample client/server apps in Go to demonstrate SDK usage.

##### Pull additional insights from README
- Emphasizes “fast, reactive, in-memory” – suggesting an event-driven, subscription-based architecture.
- Promotes Docker usage, SDKs, and CLI integration—these are external clients.

##### Components & relationships
- External Clients (CLI, SDKs, curl)
- TCP Network Layer (internal/server/ironhawk)
- I/O Multiplexer (internal/iomultiplexer)
- Command Parser & Dispatcher (internal/cmd + cmd/root.go)
- Evaluation Engine (internal/eval)
- In-Memory Storage Engine (internal/store)
- Persistence Layer (AOF & WAL in internal/store and internal/wal)
- Shard Manager & Shard Threads (internal/shard*, internal/shardmanager, internal/shardthread)
- Watch Manager (internal/watchmanager)
- Observability & Logging (internal/observability, internal/logger)
- Documentation Site (docs/ – Astro static site generator)

1. Clients → TCP Network Layer
2. Network Layer → I/O Multiplexer → Worker Thread
3. Worker Thread → Command Parser → Dispatcher
4. Dispatcher → Evaluation Engine
5. Evaluation Engine ↔ In-Memory Storage
6. On writes: Storage → Persistence Layer (WAL/AOF)
7. On data changes: Storage → Watch Manager → push notifications → Clients
8. Shard Manager routes commands to Shard Threads for per‐partition processing
9. Observability hooks capture metrics/traces throughout pipeline

c. Architectural patterns & principles:
- Reactor pattern (network I/O multiplexing)
- Sharding (concurrent partitioned in-memory data structures)
- Observer pattern (reactive subscriptions/watch manager)
- Write-ahead logging for durability
- Modular Go packages (separation of concerns, internal packages)

d. Key technologies & libraries:
- Go (modules, `go.mod`)
- Netpoll: epoll (Linux) & kqueue (Darwin) abstractions
- Protobuf (WAL definitions)
- Zerolog for structured logging
- Astro for static docs site
- Docker, Makefile, Goreleaser for builds & distributions

##### Detail
- Persistence choose (AOF vs. WAL) and where protobuf fits.
- “I/O Multiplexer” with epoll/kqueue notes.

## Diagram (Mermaid)

```mermaid
graph TB
  subgraph "External Clients & Docs"
    Clients["External Clients\n(CLI, SDKs, curl)"]:::external
    Docs["Documentation Site\n(Astro)"]:::external
  end

  subgraph "Network & I/O"
    NetLayer["TCP Network Layer\n(internal/server/ironhawk)"]:::network
    IOMux["I/O Multiplexer\n(internal/iomultiplexer)"]:::network
  end

  subgraph "Compute Layer"
    Worker["Worker Thread"]:::compute
    CmdParser["Command Parser & Dispatcher\n(internal/cmd, cmd/root.go)"]:::compute
    Eval["Evaluation Engine\n(internal/eval)"]:::compute
  end

  subgraph "Storage & Persistence"
    Storage["In-Memory Storage Engine\n(internal/store)"]:::storage
    WAL["WAL\n(internal/wal)"]:::storage
    AOF["AOF\n(internal/store/aof.go)"]:::storage
  end

  subgraph "Sharding & Watch"
    ShardMgr["Shard Manager\n(internal/shardmanager)"]:::shard
    ShardCore["Shard Core\n(internal/shard)"]:::shard
    ShardThread["Shard Threads\n(internal/shardthread)"]:::shard
    WatchMgr["Watch Manager\n(internal/watchmanager/watch_manager.go)"]:::shard
  end

  Observability["Observability\n(internal/observability)"]:::observability
  Logging["Logging\n(internal/logger)"]:::observability

  Clients --> NetLayer
  NetLayer --> IOMux
  IOMux --> Worker
  Worker --> CmdParser
  CmdParser --> Eval
  Eval <--> Storage
  Storage --> WAL
  Storage --> AOF
  Storage --> WatchMgr
  WatchMgr -->|push notifications| Clients

  CmdParser -->|route commands| ShardMgr
  ShardMgr --> ShardThread
  ShardThread --> Eval
  ShardCore --> ShardThread

  NetLayer -.-> Observability
  IOMux -.-> Observability
  Worker -.-> Observability
  CmdParser -.-> Observability
  Eval -.-> Observability
  Storage -.-> Observability

  NetLayer -.-> Logging
  IOMux -.-> Logging
  Worker -.-> Logging
  CmdParser -.-> Logging
  Eval -.-> Logging
  Storage -.-> Logging

  click NetLayer "https://github.com/dicedb/dice/tree/master/internal/server/ironhawk"
  click IOMux "https://github.com/dicedb/dice/tree/master/internal/iomultiplexer"
  click CmdParser "https://github.com/dicedb/dice/tree/master/internal/cmd"
  click CmdParser "https://github.com/dicedb/dice/blob/master/cmd/root.go"
  click Eval "https://github.com/dicedb/dice/tree/master/internal/eval"
  click Storage "https://github.com/dicedb/dice/tree/master/internal/store"
  click WAL "https://github.com/dicedb/dice/tree/master/internal/wal"
  click AOF "https://github.com/dicedb/dice/blob/master/internal/store/aof.go"
  click ShardMgr "https://github.com/dicedb/dice/tree/master/internal/shardmanager"
  click ShardCore "https://github.com/dicedb/dice/tree/master/internal/shard"
  click ShardThread "https://github.com/dicedb/dice/tree/master/internal/shardthread"
  click WatchMgr "https://github.com/dicedb/dice/blob/master/internal/watchmanager/watch_manager.go"
  click Observability "https://github.com/dicedb/dice/tree/master/internal/observability"
  click Logging "https://github.com/dicedb/dice/tree/master/internal/logger"
  click Docs "https://github.com/dicedb/dice/tree/master/docs/"

  classDef external fill:#d3d3d3,stroke:#333;
  classDef network fill:#bbdefb,stroke:#333;
  classDef compute fill:#c8e6c9,stroke:#333;
  classDef storage fill:#ffe0b2,stroke:#333;
  classDef shard fill:#e1bee7,stroke:#333;
  classDef observability fill:#f5f5f5,stroke:#333;
```
