How ggrandes/kvstore works
Project type and purpose
• This is an open-source Java library (KVStore) that provides an embeddable key-value store API.
• It implements B+Tree–based maps in memory and on disk (with fixed‐length keys/values), plus auxiliary structures (hash maps, bitsets, stacks) and storage back ends (block store, stream store).
File structure
• Maven project root (pom.xml) with Eclipse settings; no web frontend or standalone server.
• src/main/java/org/javastack/kvstore:
• Core API: KVStoreFactory, Options, Example
• Holders (serialization abstractions for Boolean, Int, Long, String, etc.)
• I/O layer: FileBlockStore, FileStreamStore, FileLongStore, StringSerializer
• Pools: BufferStacker, StringPool
• Structures:
– btree (BplusTreeMemory, BplusTreeFile, InternalNode, LeafNode)
– hash (IntHashMap, FixedIntHashMap, IntLinkedHashMap, WeakSet)
– bitset (SimpleBitSet)
– set, stack
• Utils: CRC32, CRC32C, GenericFactory, PrimeFinder, Check64bitsJVM
• src/main/java/org/javastack/kvstore/test: benchmark code
README for additional insights
• Describes features: memory vs. disk mode, mmap support, redo log, recovery, caching, file locking.
• Usage example shows how client code uses KVStoreFactory + Options to open/clear/put/get/iterate a BplusTreeFile.
a. Main components
• Client Application (user code / Example.java)
• API Layer: KVStoreFactory + Options
• Storage Abstractions:
– In-Memory Tree: BplusTreeMemory
– On-Disk Tree: BplusTreeFile
• Underlying persistence engines:
– BlockStore (FileBlockStore, FileLongStore)
– StreamStore (FileStreamStore)
• Caching & Recovery subsystem (node cache, redo log, recovery routines)
• Data-serialization layer: HolderSerializable / DataHolder + concrete Holder classes
• Support structures: HashMap, BitSet, Set, Stack implementations
• Pools: BufferStacker, StringPool
• Utilities: CRC32*, GenericFactory, PrimeFinder, etc.
b. Relationships & interactions
• Client → KVStoreFactory.createTree*(opts) → returns BplusTreeMemory or BplusTreeFile
• BplusTreeFile uses FileBlockStore / FileStreamStore for reads/writes, writes to redo log, populates node cache
• BplusTreeMemory manipulates Node objects directly in JVM heap
• Both trees call into HolderSerializable implementations to marshal/unmarshal keys & values
• Structures layer (hash maps, bitsets) can be used internally or by client
• Pools supply buffers/strings to I/O layer and structures for reuse
• Utils provide CRC checks, prime numbers, reflection‐based factory support
c. Architectural patterns & design principles
• Factory pattern (KVStoreFactory, GenericFactory) for pluggable back ends
• Strategy pattern for serialization (HolderSerializable) and storage engines (BlockStore vs. StreamStore)
• Layered architecture: API → Structures → I/O → Persistence → OS Disk
• Modular separation: core tree logic vs. I/O vs. serialization vs. utilities
d. Relevant technologies/frameworks
• Java (NIO for mmap support)
• Maven build system
• Log4J (logging)
• File locking via java.nio.channels.FileLock
Detailed capture of essential elements
• Explicitly show HolderSerializable interface feeding into B+Tree nodes and I/O streams
• Both fixed‐length (BlockStore) and variable‐length (StreamStore) paths