# wiredtiger/wiredtiger architecture

> Architecture diagram of the GitHub repository wiredtiger/wiredtiger, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wiredtiger/wiredtiger
- Repository: https://github.com/wiredtiger/wiredtiger

## Overview

- API/Connection Interface: At the top layer, the storage engine exposes its API (see “src/include/api.h”, “src/conn/conn_api.c”, etc.). This is where the application or upper layers interface with WiredTiger.
- Session and Transaction Management: Modules in “src/session”, “src/txn”, and “src/conf” constitute the management of sessions, transactions, and configurations.
- Data Storage & B-tree Structures: The “src/btree” and “src/schema” directories implement the core data structures. They handle B-tree page handling, row and column key/value storage, and schema management.
- Block and Cache Management: The “src/block” and “src/block_cache” directories show how data is abstracted as blocks and how caching, chunk-caches, and buffer management are implemented.
- Logging and Durability: Modules under “src/log” (with related Doxygen files in “src/docs”) indicate the logging subsystem that maintains durability, supports checkpoints, and handles recovery.
- Checkpoint, Recovery, and Rollback: The “src/checkpoint”, “src/rollback_to_stable”, and parts of “src/reconcile” manage periodic persistence of data and recovery consistency.
- Compression and Encryption Plugins: The “ext/compressors” and “ext/encryptors” directories indicate a plugin-style architecture where multiple algorithms can be selected at build time or dynamically configured.
- Storage Extensions and Tiered Storage: The “ext/storage_sources” (with subdirectories like “azure_store”, “gcp_store”, “s3_store”) and the “src/tiered” module suggest external storage integrations and tiered storage support.
- Utility and Support Functions: Finally, the “src/support” and “src/utilities” directories include miscellaneous functions for error handling, configuration parsing, and internal diagnostics.

##### Component Relationships and Data Flow
- Session/Connection Layer: Show a “Session Manager” component that manages connections and converts API calls into lower-level operations.
– B-tree / Schema for data manipulation.
– Block I/O and Cache Manager that manage on-disk representation.
– Logging subsystem that records changes for durability.
- Checkpoint/Recovery and Rollback: Illustrate how the logging component feeds into the checkpoint module and how rollback/recovery components interact with B-tree structures to ensure consistency.
- Utilities: Optionally, add a lower support layer that provides auxiliary functions (error processing, configuration, multi-threading support) that are used by multiple modules.

##### Architectural Patterns and Design Decisions
- Layered/Modular Architecture: Indicate the layering from the external API down to the low-level disk access and plugin integrations.
- Plug-in Mechanism: Emphasize how compression, encryption, and tiered storage components are separate from core logic, making WiredTiger highly modular.
- Concurrent and Fault-tolerant Design: Note how session management, logging, and checkpoint modules collaborate to ensure transactional integrity and data durability.

## Diagram (Mermaid)

```mermaid
graph TD
    C["Client Application"]:::client

    subgraph "API/Connection Interface"
        A1["api.h"]:::interface
        A2["conn_api.c"]:::interface
    end

    subgraph "Session & Transaction Management"
        S1["Session"]:::session
        S2["Transaction"]:::session
        S3["Configuration"]:::session
    end

    subgraph "Data Storage & B-tree Structures"
        D1["B-tree"]:::storage
        D2["Schema"]:::storage
    end

    subgraph "Block and Cache Management"
        B1["Block"]:::block
        B2["Block Cache"]:::block
    end

    subgraph "Logging and Durability"
        L1["Logging"]:::logging
        L2["Documentation"]:::logging
    end

    subgraph "Checkpoint, Recovery, and Rollback"
        C1["Checkpoint"]:::recovery
        C2["Rollback"]:::recovery
        C3["Reconcile"]:::recovery
    end

    subgraph "Compression and Encryption Plugins"
        P1["Compressors"]:::plugin
        P2["Encryptors"]:::plugin
    end

    subgraph "Storage Extensions and Tiered Storage"
        T1["Storage Sources"]:::plugin
        T2["Tiered Storage"]:::plugin
    end

    subgraph "Utility and Support Functions"
        U1["Support"]:::utility
        U2["Utilities"]:::utility
    end

    C --> A1
    C --> A2
    A1 --> S1
    A2 --> S1
    S1 --> D1
    S1 --> B1
    S1 --> B2
    S1 --> L1
    L1 --> L2
    L1 --> C1
    L1 --> C2
    L1 --> C3
    C1 --> D1
    D1 --> P1
    D1 --> P2
    B2 --> T1
    B2 --> T2
    S1 --> U1
    D1 --> U2
    L1 --> U2

    click A1 "https://github.com/wiredtiger/wiredtiger/blob/develop/src/include/api.h"
    click A2 "https://github.com/wiredtiger/wiredtiger/blob/develop/src/conn/conn_api.c"
    click S1 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/session"
    click S2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/txn"
    click S3 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/conf"
    click D1 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/btree"
    click D2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/schema"
    click B1 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/block"
    click B2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/block_cache"
    click L1 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/log"
    click L2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/docs"
    click C1 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/checkpoint"
    click C2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/rollback_to_stable"
    click C3 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/reconcile"
    click P1 "https://github.com/wiredtiger/wiredtiger/tree/develop/ext/compressors"
    click P2 "https://github.com/wiredtiger/wiredtiger/tree/develop/ext/encryptors"
    click T1 "https://github.com/wiredtiger/wiredtiger/tree/develop/ext/storage_sources"
    click T2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/tiered"
    click U1 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/support"
    click U2 "https://github.com/wiredtiger/wiredtiger/tree/develop/src/utilities"

    classDef client fill:#FAD7A0,stroke:#E67E22,stroke-width:2px;
    classDef interface fill:#AED6F1,stroke:#2471A3,stroke-width:2px;
    classDef session fill:#85C1E9,stroke:#1F618D,stroke-width:2px;
    classDef storage fill:#82E0AA,stroke:#28B463,stroke-width:2px;
    classDef block fill:#F9E79F,stroke:#D4AC0D,stroke-width:2px;
    classDef logging fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef recovery fill:#E6B0AA,stroke:#922B21,stroke-width:2px;
    classDef plugin fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px;
    classDef utility fill:#D5DBDB,stroke:#7F8C8D,stroke-width:2px;
```
