How simerplaha/swaydb works
Project type and purpose
• This is SwayDB, a modular, high-performance persistent and in-memory key-value storage engine for the JVM. It is not a web service but a library composed of multiple SBT modules that provide APIs (Map, Set, Queue, MultiMap, Transactions, Streams, TTL, etc.), a non-blocking core engine, pluggable persistence layers, compression, caching and extension points for functional and reactive use-cases.
File structure
• Top-level modules (visible in the root):
• actor: orchestration, scheduling, asynchronous hooks
• configs: default and builder configurations
• core-*: building blocks of the engine (cache, compression, interop, queue, skiplist, series, performance tests, tools)
• core: main engine entry point (CoreInitializer, CoreState, Compactor, Segment manager, File sweeper)
• effect: IO abstraction layer
• macros: codegen utilities
• serializers, slice, stream: serialization, slice data structure, streaming APIs
• stress, core-stress: stress testing harnesses
• utils: common utilities
• x-interop-*: adapters for Boopickle, Cats-Effect, Monix, ZIO
• swaydb, swaydb-java: user-facing Scala & Java APIs
• testkit, docs, project, build.sbt, version.sbt, .github, LICENSE, README.md
• Patterns:
• Layered architecture: API layer (swaydb/, swaydb-java/) → Effect/interop → Core engine (core/, core-*) → Low-level modules (file I/O, segment, block, cache, compression, skiplist, queue, series) → OS/disk
• Actor-based scheduling and thread pools for non-blocking operations
• SBT multi-project build orchestrating module dependencies
README for additional insights
• Highlights:
• “non-blocking core” with configurable blocking/reactive APIs
• Single/multi-disk persistence, in-memory, or eventually persistent modes
• Streams, TTL, Range operations, Configurable compression (LZ4 & Snappy)
• Extensibility: build new data structures, custom serialisers, builders
Details to capture
– Thread pools: IO, compaction, scheduler
– Cache layers vs on-disk segments vs logs
– Compaction workflow: assignment → throttle → merge → write
– Extensibility points: new data structures, custom serializers, custom config