# ggrandes/kvstore architecture

> Architecture diagram of the GitHub repository ggrandes/kvstore, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ggrandes/kvstore
- Repository: https://github.com/ggrandes/kvstore

## Overview

##### Project type and purpose
- This is an open-source Java library (KVStore) that provides an embeddable key-value store API.
- It implements B+Tree–based maps in memory and on disk (with fixed‐length keys/values), plus auxiliary structures (hash maps, bitsets, stacks) and storage back ends (block store, stream store).

##### File structure
- Maven project root (pom.xml) with Eclipse settings; no web frontend or standalone server.
- src/main/java/org/javastack/kvstore:
- Core API: KVStoreFactory, Options, Example
- Holders (serialization abstractions for Boolean, Int, Long, String, etc.)
- I/O layer: FileBlockStore, FileStreamStore, FileLongStore, StringSerializer
- Pools: BufferStacker, StringPool
- Structures:
– btree (BplusTreeMemory, BplusTreeFile, InternalNode, LeafNode)
– hash (IntHashMap, FixedIntHashMap, IntLinkedHashMap, WeakSet)
– bitset (SimpleBitSet)
– set, stack
- Utils: CRC32, CRC32C, GenericFactory, PrimeFinder, Check64bitsJVM
- src/main/java/org/javastack/kvstore/test: benchmark code

##### README for additional insights
- Describes features: memory vs. disk mode, mmap support, redo log, recovery, caching, file locking.
- Usage example shows how client code uses KVStoreFactory + Options to open/clear/put/get/iterate a BplusTreeFile.

a. Main components
- Client Application (user code / Example.java)
- API Layer: KVStoreFactory + Options
- Storage Abstractions:
– In-Memory Tree: BplusTreeMemory
– On-Disk Tree: BplusTreeFile
- Underlying persistence engines:
– BlockStore (FileBlockStore, FileLongStore)
– StreamStore (FileStreamStore)
- Caching & Recovery subsystem (node cache, redo log, recovery routines)
- Data-serialization layer: HolderSerializable / DataHolder + concrete Holder classes
- Support structures: HashMap, BitSet, Set, Stack implementations
- Pools: BufferStacker, StringPool
- Utilities: CRC32*, GenericFactory, PrimeFinder, etc.
b. Relationships & interactions
- Client → KVStoreFactory.createTree*(opts) → returns BplusTreeMemory or BplusTreeFile
- BplusTreeFile uses FileBlockStore / FileStreamStore for reads/writes, writes to redo log, populates node cache
- BplusTreeMemory manipulates Node objects directly in JVM heap
- Both trees call into HolderSerializable implementations to marshal/unmarshal keys & values
- Structures layer (hash maps, bitsets) can be used internally or by client
- Pools supply buffers/strings to I/O layer and structures for reuse
- Utils provide CRC checks, prime numbers, reflection‐based factory support
c. Architectural patterns & design principles
- Factory pattern (KVStoreFactory, GenericFactory) for pluggable back ends
- Strategy pattern for serialization (HolderSerializable) and storage engines (BlockStore vs. StreamStore)
- Layered architecture: API → Structures → I/O → Persistence → OS Disk
- Modular separation: core tree logic vs. I/O vs. serialization vs. utilities
d. Relevant technologies/frameworks
- Java (NIO for mmap support)
- Maven build system
- Log4J (logging)
- File locking via java.nio.channels.FileLock

##### Detailed capture of essential elements
- Explicitly show HolderSerializable interface feeding into B+Tree nodes and I/O streams
- Both fixed‐length (BlockStore) and variable‐length (StreamStore) paths

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client/Test Code"
        direction TB
        Example["Client Application<br/>(Example.java)"]:::client
    end

    %% API Layer
    subgraph "API Layer"
        direction TB
        KVFactory["KVStoreFactory"]:::api
        Options["Options"]:::api
    end

    %% Tree Layer
    subgraph "Tree Implementations"
        direction TB
        BPTreeMem["BplusTreeMemory"]:::structure
        BPTreeFile["BplusTreeFile"]:::structure
        BPTreeIface["BplusTree (interface)"]:::structure
        InternalNode["InternalNode"]:::structure
        LeafNode["LeafNode"]:::structure
        NodeBase["Node"]:::structure
    end

    %% Persistence Engines
    subgraph "Persistence Engines"
        direction TB
        FileBlock["FileBlockStore"]:::persistence
        FileLong["FileLongStore"]:::persistence
        FileStream["FileStreamStore"]:::persistence
        IOConst["Constants"]:::persistence
        StrSerializer["StringSerializer"]:::persistence
    end

    %% Serialization Layer
    subgraph "Serialization Layer"
        direction TB
        HolderIface["HolderSerializable"]:::util
        DataHolder["DataHolder"]:::util
        BoolHolder["BooleanHolder"]:::util
        IntHolder["IntHolder"]:::util
        LongHolder["LongHolder"]:::util
        StringHolder["StringHolder"]:::util
    end

    %% Support Structures
    subgraph "Support Structures"
        direction TB
        IntHash["IntHashMap"]:::structure
        FixedIntHash["FixedIntHashMap"]:::structure
        LinkedHash["IntLinkedHashMap"]:::structure
        WeakSet["WeakSet"]:::structure
        BitSet["SimpleBitSet"]:::structure
        SortedSet["SortedIntArraySet"]:::structure
        IntStack["IntStack"]:::structure
        ObjStack["ObjectStack"]:::structure
    end

    %% Pools
    subgraph "Pools"
        direction TB
        BufferPool["BufferStacker"]:::util
        StringPool["StringPool"]:::util
    end

    %% Utilities
    subgraph "Utilities"
        direction TB
        CRC32["CRC32"]:::util
        CRC32C["CRC32C"]:::util
        CheckJVM["Check64bitsJVM"]:::util
        GenFactory["GenericFactory"]:::util
        PrimeFinder["PrimeFinder"]:::util
        HexStrings["HexStrings"]:::util
    end

    %% Test Benchmarks
    subgraph "Benchmark Tests"
        direction TB
        BenchDisk["BenchMarkDiskStore"]:::client
        BenchMem["BenchMarkMemoryStructures"]:::client
    end

    %% Relationships
    Example -->|uses| KVFactory
    KVFactory -->|uses| Options
    KVFactory -->|creates| BPTreeMem
    KVFactory -->|creates| BPTreeFile

    BPTreeMem -->|implements| BPTreeIface
    BPTreeFile -->|implements| BPTreeIface
    BPTreeFile -->|reads/writes| FileBlock
    BPTreeFile -->|reads/writes| FileStream
    BPTreeFile -->|reads/writes| FileLong
    BPTreeFile -->|uses| IOConst
    BPTreeFile -->|uses| StrSerializer

    BPTreeMem -->|uses| HolderIface
    BPTreeFile -->|uses| HolderIface
    HolderIface -->|extends| DataHolder
    DataHolder -->|concrete| BoolHolder
    DataHolder -->|concrete| IntHolder
    DataHolder -->|concrete| LongHolder
    DataHolder -->|concrete| StringHolder

    BPTreeIface --> InternalNode
    BPTreeIface --> LeafNode
    InternalNode --> NodeBase
    LeafNode --> NodeBase

    BPTreeMem -->|may use| IntHash
    BPTreeMem -->|may use| FixedIntHash
    BPTreeMem -->|may use| LinkedHash
    BPTreeMem -->|may use| WeakSet
    BPTreeMem -->|may use| BitSet
    BPTreeMem -->|may use| SortedSet
    BPTreeMem -->|may use| IntStack
    BPTreeMem -->|may use| ObjStack

    BufferPool -->|supplies buffers| FileBlock
    BufferPool -->|supplies buffers| FileStream
    BufferPool -->|supplies buffers| FileLong
    BufferPool -->|supplies buffers| IntHash

    StringPool -->|supplies strings| StrSerializer
    StringPool -->|supplies strings| StringHolder

    CRC32 -->|used by| FileBlock
    CRC32C -->|used by| FileBlock
    CheckJVM -->|startup check| KVFactory
    GenFactory -->|factory support| KVFactory
    PrimeFinder -->|used by| BPTreeMem
    HexStrings -->|util| KVFactory

    BenchDisk -->|benchmarks| BPTreeFile
    BenchDisk -->|benchmarks| FileBlock
    BenchDisk -->|benchmarks| FileStream

    BenchMem -->|benchmarks| BPTreeMem
    BenchMem -->|benchmarks| IntHash
    BenchMem -->|benchmarks| BitSet

    %% Click Events
    click Example "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/Example.java"
    click KVFactory "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/KVStoreFactory.java"
    click Options "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/Options.java"
    click BPTreeMem "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/btree/BplusTreeMemory.java"
    click BPTreeFile "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/btree/BplusTreeFile.java"
    click BPTreeIface "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/btree/BplusTree.java"
    click InternalNode "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/btree/InternalNode.java"
    click LeafNode "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/btree/LeafNode.java"
    click NodeBase "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/btree/Node.java"
    click FileBlock "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/io/FileBlockStore.java"
    click FileLong "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/io/FileLongStore.java"
    click FileStream "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/io/FileStreamStore.java"
    click HolderIface "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/holders/HolderSerializable.java"
    click DataHolder "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/holders/DataHolder.java"
    click BoolHolder "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/holders/BooleanHolder.java"
    click IntHolder "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/holders/IntHolder.java"
    click LongHolder "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/holders/LongHolder.java"
    click StringHolder "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/holders/StringHolder.java"
    click IntHash "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/hash/IntHashMap.java"
    click FixedIntHash "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/hash/FixedIntHashMap.java"
    click LinkedHash "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/hash/IntLinkedHashMap.java"
    click WeakSet "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/hash/WeakSet.java"
    click BitSet "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/bitset/SimpleBitSet.java"
    click SortedSet "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/set/SortedIntArraySet.java"
    click IntStack "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/stack/IntStack.java"
    click ObjStack "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/structures/stack/ObjectStack.java"
    click BufferPool "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/pool/BufferStacker.java"
    click StringPool "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/pool/StringPool.java"
    click CRC32 "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/utils/CRC32.java"
    click CRC32C "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/utils/CRC32C.java"
    click CheckJVM "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/utils/Check64bitsJVM.java"
    click GenFactory "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/utils/GenericFactory.java"
    click PrimeFinder "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/utils/PrimeFinder.java"
    click HexStrings "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/utils/HexStrings.java"
    click IOConst "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/io/Constants.java"
    click StrSerializer "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/io/StringSerializer.java"
    click BenchDisk "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/test/BenchMarkDiskStore.java"
    click BenchMem "https://github.com/ggrandes/kvstore/blob/main/src/main/java/org/javastack/kvstore/test/BenchMarkMemoryStructures.java"

    %% Styles
    classDef api fill:#D0E4F5,stroke:#0066CC,color:#000
    classDef structure fill:#E2F5D0,stroke:#339900,color:#000
    classDef persistence fill:#F5E0D0,stroke:#CC6600,color:#000
    classDef util fill:#E7D0F5,stroke:#6600CC,color:#000
    classDef client fill:#DDDDDD,stroke:#888888,color:#000
```
