# mongodb-partners/mongo-rocks architecture

> Architecture diagram of the GitHub repository mongodb-partners/mongo-rocks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mongodb-partners/mongo-rocks
- Repository: https://github.com/mongodb-partners/mongo-rocks

## Overview

##### Project Type and Purpose
- This is a native C++ “storage engine” plugin for MongoDB, named “mongo-rocks,” which implements MongoDB’s StorageEngine interface on top of Facebook’s RocksDB.
- It is designed to be linked into the `mongo` server (`mongod`) under `mongo/src/mongo/db/modules/rocks`, then built via SCons alongside the main MongoDB codebase.

##### File‐Structure Analysis
- Top-level build and config:
- BUILD.md, INSTALL.md references, SConscript, build.py, resmoke.sh, resmokelist
- Source directory (`src/`):
- Core engine entry point: `rocks_engine.cpp` / `.h`
- Manager classes: durability, compaction scheduler, snapshot manager, counter manager
- Storage abstractions: index, oplog manager, record store (mocks and mongod variant)
- Transactional layer (`src/totdb/`): a thin wrapper around `RocksDB::TransactionDB`
- Options & parameters (IDL files): `rocks_global_options.idl`, `rocks_parameters.idl`
- Test coverage: `*_test.cpp` files validating each major component
- Config and lint: `.clang-format`, `.gitignore`, `.arcconfig`

##### README Insights
- Describes build prerequisites (RocksDB library, SCons variables)
- Explains how to integrate this module into the MongoDB repository

a. Main Components
- MongoDB Server Process (`mongod`)
- StorageEngine Plugin (shared library) – “RocksEngine”
- Internal Plugin Subcomponents:
– DurabilityManager
– SnapshotManager
– CompactionScheduler
– CounterManager
– GlobalOptions & Parameters (IDL-generated)
– Index and OplogManager
– RecordStore (mock and mongod‐specific)
– RecoveryUnit
– totDb Module (TransactionDB wrapper)
- External Dependency: RocksDB Core Library
- Test Harness (Resmoke / unit tests)
b. Relationships & Data Flows
1. Client/Driver → MongoDB API → MongoDB Internals
2. MongoDB writes/reads → invokes StorageEngine API on RocksEngine
3. RocksEngine dispatches calls to subcomponents:
– Write path: RecordStore → RecoveryUnit → DurabilityManager → RocksDB
– Read path: RecordStore / Index → RocksDB
– Transaction path: totDb → RocksDB::TransactionDB
– Background tasks: CompactionScheduler & SnapshotManager → RocksDB
4. GlobalOptions & Parameters feed configuration into each manager at startup
5. Unit tests (`*_test.cpp`) drive individual subcomponents in isolation
c. Architectural Patterns & Principles
- Plugin architecture via StorageEngine interface
- Single‐Responsibility managers (durability, compaction, snapshot)
- Transactional abstraction layered on RocksDB transactions
- IDL‐driven options for schema‐safe configuration
- Separation of concerns: host interface vs. plugin internals vs. external library
d. Key Technologies & Frameworks
- C++17 (or later)
- SCons build system
- RocksDB (C++ library)
- MongoDB internal StorageEngine APIs
- IDL macros for option parsing
- Resmoke test harness

##### Essential Details
- Don’t merely list files; group them into runtime components
- How global options propagate into managers at startup
- Sequence: initialization → operation → shutdown

## Diagram (Mermaid)

```mermaid
graph TB
    %% Host Layer
    subgraph "MongoDB Core" 
        direction TB
        Client["Client/Driver"]:::host
        MongoAPI["MongoDB API"]:::host
        MongoCore["mongod Process"]:::host
    end

    %% Plugin Boundary
    MongoCore -->|"invoke StorageEngine"| RocksEngineEntry

    %% Plugin Layer
    subgraph "RocksEngine Plugin" 
        direction TB
        RocksEngineEntry["RocksEngine"]:::plugin
        subgraph "Durability & Recovery"
            direction TB
            Durability["DurabilityManager"]:::plugin
            Recovery["RecoveryUnit"]:::plugin
        end
        subgraph "Background Scheduling"
            direction TB
            Snapshot["SnapshotManager"]:::plugin
            Compaction["CompactionScheduler"]:::plugin
        end
        Counter["CounterManager"]:::plugin
        subgraph "Storage Abstractions"
            direction TB
            RecordStore["RecordStore"]:::plugin
            Index["Index & OplogManager"]:::plugin
            Oplog["OplogManager"]:::plugin
        end
        subgraph "Transaction Layer"
            direction TB
            totDb["totDb TransactionDB Wrapper"]:::plugin
        end
        subgraph "Configuration"
            direction TB
            GlobalOpts["GlobalOptions"]:::plugin
            Params["Parameters"]:::plugin
        end
    end

    %% External Layer
    RocksDB["RocksDB Library"]:::external

    %% Test Harness
    subgraph "Test Harness" 
        direction TB
        Tests["Unit & Resmoke Tests"]:::test
    end

    %% Data Flows
    Client --> MongoAPI
    MongoAPI --> MongoCore
    RocksEngineEntry -->|"init config"| GlobalOpts
    RocksEngineEntry -->|"init config"| Params
    RocksEngineEntry -->|"call record operations"| RecordStore
    RecordStore -->|"reads/writes"| RocksDB
    Index -->|"index lookup"| RocksDB
    Oplog -->|"oplog writes"| RocksDB
    RecordStore --> Recovery
    Recovery --> Durability
    Durability -->|"writeBatch() / commitUnitOfWork()"| RocksDB
    totDb -->|"transaction operations"| RocksDB
    Snapshot -->|"createSnapshot()"| RocksDB
    Compaction -->|"scheduleCompaction()"| RocksDB
    Counter -->|"increment/decrement"| RocksDB

    %% Lifecycle dependencies
    GlobalOpts -.-> Durability
    GlobalOpts -.-> Snapshot
    GlobalOpts -.-> Compaction
    Params -.-> Durability
    Params -.-> Snapshot

    %% Tests invoke subcomponents
    Tests --> Durability
    Tests --> Recovery
    Tests --> Snapshot
    Tests --> Compaction
    Tests --> Counter
    Tests --> RecordStore
    Tests --> Index
    Tests --> totDb

    %% Click Events
    click RocksEngineEntry "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_engine.cpp"
    click Durability "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_durability_manager.cpp"
    click Snapshot "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_snapshot_manager.cpp"
    click Compaction "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_compaction_scheduler.cpp"
    click Counter "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_counter_manager.cpp"
    click GlobalOpts "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_global_options.idl"
    click Params "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_parameters.idl"
    click Index "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_index.cpp"
    click Oplog "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_oplog_manager.cpp"
    click RecordStore "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_record_store.cpp"
    click Recovery "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_recovery_unit.cpp"
    click totDb "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/totdb/totransaction_db.h"
    click Tests "https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_engine.cpp"

    %% Styles
    classDef host fill:#ccffcc,stroke:#333,stroke-width:1px
    classDef plugin fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef external fill:#ffddcc,stroke:#333,stroke-width:1px
    classDef test fill:#eeeeee,stroke:#333,stroke-width:1px
```
