# thrumdev/nomt architecture

> Architecture diagram of the GitHub repository thrumdev/nomt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thrumdev/nomt
- Repository: https://github.com/thrumdev/nomt

## Overview

##### Main Components
- Core Database Engine – divide this into two main internal stores:
⁃ Beatree: A B-Tree–variant structure optimized for rapid random access of high-entropy key-value pairs.
⁃ Bitbox: A custom sparse binary Merkle tree stored on-disk, using a hashtable format designed for SSD optimization.
- API Layer – the key-value API that handles “Sessions” (i.e. the many-readers-one-writer transaction model).
- Supporting Libraries (from the “core” directory) – include components like hashing, page management, trie operations, proof generation, and concurrency control.
- Benchmarking and Test Harnesses:
⁃ Benchtop: A benchmarking tool housed in its own subcrate.
⁃ Fuzz: A suite for fuzz testing the API and internal methods.
⁃ Torture: An extensive testing suite for stress-testing the system.
- Experimental and Auxiliary Tools:
⁃ Trickfs/Trickmnt: A FUSE filesystem integration for deeper testing and experimental mounting of the storage layer.

- A clear separation between the external API (which provides a key-value API and “Session” based access) and the underlying database engine.
⁃ From client requests going into the API layer.
⁃ API layer interfacing with both Beatree and Bitbox in parallel.
- Feedback loops where needed—for instance, from the core library (managing pages, proofs, updates) back into the API for session coordination.

##### Architectural Patterns and Design Principles
- Dual-store design pattern (parallel stores: Beatree and Bitbox) that encapsulates the separation between raw key-value handling and merklized data integrity.
- “many-readers-one-writer” session management approach as part of the API’s transactional design.
- Use of modular testing and benchmarking as separate subcomponents that validate and stress the main engine.
- Mention the integration with modern hardware (SSD, NVMe, Linux as the primary target) as a performance enabler.

##### Relevant Technologies, Frameworks, and Languages
- Rust is the language of implementation.
- Rust components such as Cargo (as seen in the project’s Cargo.toml files) and the areas where different subcrates (nomt, core, benchtop, etc.) are used.
- Where asynchronous I/O (with Linux’s io-uring API) is being leveraged for performance.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Interaction Layer
    subgraph "External Interaction Layer"
        client["External Client"]:::external
    end

    %% API Layer
    subgraph "API Layer"
        api["API - Session Manager (Key-Value)"]:::api
    end

    %% Core Processing Layer
    subgraph "Core Processing Layer"
        subgraph "Core Database Engine"
            beatree["Database Engine – Beatree"]:::db
            bitbox["Database Engine – Bitbox"]:::db
        end
        libs["Core Libraries (Hasher, Page, Trie, Proof)"]:::lib
    end

    %% Testing/Benchmarking/Experimental Layer
    subgraph "Testing/Benchmarking/Experimental Layer"
        subgraph "Testing & Benchmarking"
            benchtop["Benchmarking Tool – Benchtop"]:::test
            fuzz["Fuzz Testing Suite"]:::test
            torture["Torture Testing Suite"]:::test
        end
        subgraph "Experimental Tools"
            trickfs["Experimental Filesystem – Trickfs/Trickmnt"]:::experimental
        end
        excomp["Examples"]:::external2
        docs["Documentation"]:::external2
    end

    %% Data flow connections (vertical orientation)
    client -->|"request"| api

    api -->|"writes/reads"| beatree
    api -->|"writes/reads"| bitbox

    beatree -->|"response"| api
    bitbox -->|"response"| api

    libs -->|"support"| beatree
    libs -->|"support"| bitbox
    libs -->|"session feedback"| api

    benchtop ---|"performance test"| beatree
    benchtop ---|"performance test"| bitbox

    fuzz ---|"API fuzzing"| api
    torture ---|"stress test"| api

    trickfs ---|"integration"| beatree
    trickfs ---|"integration"| bitbox

    client ---|"reference"| excomp
    client ---|"reference"| docs

    %% Click Events
    click api "https://github.com/thrumdev/nomt/blob/master/nomt/src/lib.rs"
    click beatree "https://github.com/thrumdev/nomt/tree/master/nomt/src/beatree"
    click bitbox "https://github.com/thrumdev/nomt/tree/master/nomt/src/bitbox"
    click libs "https://github.com/thrumdev/nomt/tree/master/core/"
    click benchtop "https://github.com/thrumdev/nomt/tree/master/benchtop/"
    click fuzz "https://github.com/thrumdev/nomt/tree/master/fuzz/"
    click torture "https://github.com/thrumdev/nomt/tree/master/torture/"
    click trickfs "https://github.com/thrumdev/nomt/tree/master/trickfs/"
    click excomp "https://github.com/thrumdev/nomt/tree/master/examples/"
    click docs "https://github.com/thrumdev/nomt/tree/master/docs/"

    %% Styles
    classDef external fill:#f9c,stroke:#333,stroke-width:2px;
    classDef external2 fill:#f96,stroke:#333,stroke-width:2px;
    classDef api fill:#bbf,stroke:#333,stroke-width:2px;
    classDef db fill:#cfc,stroke:#333,stroke-width:2px;
    classDef lib fill:#fcf,stroke:#333,stroke-width:2px;
    classDef test fill:#ff9,stroke:#333,stroke-width:2px;
    classDef experimental fill:#9ff,stroke:#333,stroke-width:2px;
```
