# cshekharsharma/universum architecture

> Architecture diagram of the GitHub repository cshekharsharma/universum, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cshekharsharma/universum
- Repository: https://github.com/cshekharsharma/universum

## Overview

##### Main Components
- Request Processing Engine: This includes components under the “engine” directory (e.g., commands.go, datastore.go, eviction.go, and snapshot.go) that process client commands, parse them, and route the workflow.
- Storage Engine: Split this into two parts:
- In-Memory Storage: Found under storage/memory (for TTL support and snapshot functionalities).
- Cluster and Distributed Module: Even though multi-node cluster mode is a roadmap feature, the “cluster” directory (gossip, heartbeat, partitions, etc.) should be represented as a supporting module that will eventually interact with the main engine if/when distributed functionalities are enabled.
- Configuration Management: The “config” directory holds config, defaults, TOML parsing, and validation that initialize system settings; show how this layer feeds into other components at startup.
- Compression & Crypto Libraries: Illustrate the “compression” (e.g., LZ4 implementations) and “crypto” (murmur hash) modules as supporting libraries used by the storage layer (and possibly by the network protocol or engine) to improve performance and data integrity.
- Internal Services: Such as the logger in the “internal/logger” and the RESP3 protocol handling (“resp3/resp3.go”), which indicates that responses are formatted according to the RESP3 protocol.

##### Relationships and Flows
- Bidirectional data flow where applicable (for instance, engine requesting snapshots from storage modules).
- Dependencies such as configuration feeding into both the engine initialization and storage setups.
- That the compression and crypto modules serve as libraries included by the storage components when handling data (e.g., compressing SSTable blocks or hashing keys).

##### Architectural Patterns and Design Principles
- Separation of concerns: networking layer (server), business logic layer (engine), and data persistence layer (storage).
- Use of modular design—each functionality (cluster, config, compression, crypto) is self-contained, which supports future extensibility, especially toward a distributed design.
- Worker-based multithreaded I/O engine (as mentioned in the README), so lifetime threads handling commands and leveraging worker pools should be marked as part of the engine component.
- Visible patterns like the use of interfaces (e.g., storage/interface.go) that abstract storage so that in-memory and persistent implementations can be switched or combined.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Networking Layer
    subgraph "Networking Layer"
        Client["Client (External)"]:::client
        tcpServer["TCP Server"]:::server
        resp3["RESP3 Protocol Handler"]:::resp3
    end

    %% Processing Layer
    subgraph "Processing Layer"
        engine["Command Processing Engine"]:::engine
    end

    %% Storage Layer
    subgraph "Storage Layer"
        memory["In-Memory Store"]:::memory
        lsm["LSM Persistent Store"]:::persistent
    end

    %% Supporting Modules
    subgraph "Supporting Modules"
        cluster["Cluster Module"]:::module
        config["Configuration Management"]:::module
        compression["Compression Library"]:::module
        crypto["Crypto Library"]:::module
        logger["Internal Logger"]:::module
    end

    %% Connections for Networking to Engine
    Client -->|"TCP_connection"| tcpServer
    tcpServer -->|"command_dispatch"| engine
    tcpServer -->|"formats_response"| resp3

    %% Engine to Storage interactions
    engine -->|"read/write_commands"| memory
    engine -->|"read/write_commands"| lsm

    %% Supporting Modules feeding into Engine and Storage
    config -->|"feeds_configuration"| engine
    config -->|"feeds_configuration"| memory
    config -->|"feeds_configuration"| lsm
    cluster -->|"future_integration"| engine
    compression -->|"data_compression"| lsm
    crypto -->|"data_integrity"| lsm
    engine -->|"logs_activity"| logger

    %% Styles
    classDef client fill:#b3e5fc,stroke:#0288d1,stroke-width:2px;
    classDef server fill:#4fc3f7,stroke:#01579b,stroke-width:2px;
    classDef resp3 fill:#ff8a65,stroke:#d84315,stroke-width:2px;
    classDef engine fill:#81c784,stroke:#388e3c,stroke-width:2px;
    classDef memory fill:#fff176,stroke:#f9a825,stroke-width:2px;
    classDef persistent fill:#ffd54f,stroke:#fb8c00,stroke-width:2px;
    classDef module fill:#ce93d8,stroke:#8e24aa,stroke-width:2px;

    %% Click Events
    click tcpServer "https://github.com/cshekharsharma/universum/blob/main/server/tcpserver.go"
    click tcpServer "https://github.com/cshekharsharma/universum/blob/main/server/serverutils.go"
    click resp3 "https://github.com/cshekharsharma/universum/blob/main/resp3/resp3.go"
    click engine "https://github.com/cshekharsharma/universum/blob/main/engine/commands.go"
    click engine "https://github.com/cshekharsharma/universum/blob/main/engine/datastore.go"
    click engine "https://github.com/cshekharsharma/universum/blob/main/engine/eviction.go"
    click engine "https://github.com/cshekharsharma/universum/blob/main/engine/snapshot.go"
    click engine "https://github.com/cshekharsharma/universum/blob/main/engine/workers.go"
    click memory "https://github.com/cshekharsharma/universum/blob/main/storage/memory/memstore.go"
    click memory "https://github.com/cshekharsharma/universum/blob/main/storage/memory/snapshot.go"
    click lsm "https://github.com/cshekharsharma/universum/blob/main/storage/lsm/lsmstore.go"
    click lsm "https://github.com/cshekharsharma/universum/tree/main/storage/lsm/compaction/"
    click lsm "https://github.com/cshekharsharma/universum/tree/main/storage/lsm/memtable/"
    click lsm "https://github.com/cshekharsharma/universum/tree/main/storage/lsm/sstable/"
    click lsm "https://github.com/cshekharsharma/universum/tree/main/storage/lsm/wal/"
    click cluster "https://github.com/cshekharsharma/universum/tree/main/cluster/"
    click config "https://github.com/cshekharsharma/universum/tree/main/config/"
    click compression "https://github.com/cshekharsharma/universum/tree/main/compression/"
    click crypto "https://github.com/cshekharsharma/universum/tree/main/crypto/"
    click logger "https://github.com/cshekharsharma/universum/tree/main/internal/logger/"
```
