How mongodb-partners/mongo-rocks works
Project Type and Purpose
• This is a native C++ “storage engine” plugin for MongoDB, named “mongo-rocks,” which implements MongoDB’s StorageEngine interface on top of Facebook’s RocksDB.
• It is designed to be linked into the mongo server (mongod) under mongo/src/mongo/db/modules/rocks, then built via SCons alongside the main MongoDB codebase.
File‐Structure Analysis
• Top-level build and config:
• BUILD.md, INSTALL.md references, SConscript, build.py, resmoke.sh, resmokelist
• Source directory (src/):
• Core engine entry point: rocks_engine.cpp / .h
• Manager classes: durability, compaction scheduler, snapshot manager, counter manager
• Storage abstractions: index, oplog manager, record store (mocks and mongod variant)
• Transactional layer (src/totdb/): a thin wrapper around RocksDB::TransactionDB
• Options & parameters (IDL files): rocks_global_options.idl, rocks_parameters.idl
• Test coverage: *_test.cpp files validating each major component
• Config and lint: .clang-format, .gitignore, .arcconfig
README Insights
• Describes build prerequisites (RocksDB library, SCons variables)
• Explains how to integrate this module into the MongoDB repository
a. Main Components
• MongoDB Server Process (mongod)
• StorageEngine Plugin (shared library) – “RocksEngine”
• Internal Plugin Subcomponents:
– DurabilityManager
– SnapshotManager
– CompactionScheduler
– CounterManager
– GlobalOptions & Parameters (IDL-generated)
– Index and OplogManager
– RecordStore (mock and mongod‐specific)
– RecoveryUnit
– totDb Module (TransactionDB wrapper)
• External Dependency: RocksDB Core Library
• Test Harness (Resmoke / unit tests)
b. Relationships & Data Flows
1. Client/Driver → MongoDB API → MongoDB Internals
2. MongoDB writes/reads → invokes StorageEngine API on RocksEngine
3. RocksEngine dispatches calls to subcomponents:
– Write path: RecordStore → RecoveryUnit → DurabilityManager → RocksDB
– Read path: RecordStore / Index → RocksDB
– Transaction path: totDb → RocksDB::TransactionDB
– Background tasks: CompactionScheduler & SnapshotManager → RocksDB
4. GlobalOptions & Parameters feed configuration into each manager at startup
5. Unit tests (*_test.cpp) drive individual subcomponents in isolation
c. Architectural Patterns & Principles
• Plugin architecture via StorageEngine interface
• Single‐Responsibility managers (durability, compaction, snapshot)
• Transactional abstraction layered on RocksDB transactions
• IDL‐driven options for schema‐safe configuration
• Separation of concerns: host interface vs. plugin internals vs. external library
d. Key Technologies & Frameworks
• C++17 (or later)
• SCons build system
• RocksDB (C++ library)
• MongoDB internal StorageEngine APIs
• IDL macros for option parsing
• Resmoke test harness
Essential Details
• Don’t merely list files; group them into runtime components
• How global options propagate into managers at startup
• Sequence: initialization → operation → shutdown