# apple/foundationdb architecture

> Architecture diagram of the GitHub repository apple/foundationdb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apple/foundationdb
- Repository: https://github.com/apple/foundationdb

## Overview

##### Main Components Overview

a) Client Layer:
- Multiple language bindings (C, Java, Python, Go, Ruby)
- Client API interfaces
- Transaction management

b) Server Components:
- Cluster Controller
- Commit Proxy
- Storage Servers
- Transaction Logs (TLog)
- Master
- Resolver
- Coordinators

##### Data Flow Architecture

a) Write Path:
- Client → Commit Proxy
- Commit Proxy → Resolver (conflict resolution)
- Commit Proxy → Transaction Log
- Transaction Log → Storage Servers

b) Read Path:
- Client → Storage Servers (direct reads)
- Client → Proxies (for consistent reads)

##### Key Subsystems

a) Storage Layer:
- Key-value store (RocksDB backend)
- Sharding mechanism
- Replication

b) Transaction System:
- ACID guarantees
- Versioning
- Conflict detection

c) Distribution System:
- Data distribution
- Load balancing
- Failure detection

##### Cross-Cutting Concerns
Overlays showing:

a) Monitoring & Management:
- Metrics collection
- Status reporting
- Backup systems

b) Security:
- TLS connections
- Authentication
- Authorization

##### Connection Types
Use different line styles for:
- Replication streams (thick lines)

##### Specific Implementation Details
- Actor model implementation
- Flow language usage
- Metadata management
- Tenant isolation

##### Operational Components
Supporting systems:
- Backup/restore functionality
- Blob storage integration
- Monitoring interfaces
- Configuration management

Key considerations:
- Separation between client and server components
- How the system maintains consistency across distributed components
- Failure handling and recovery paths
- Demonstrate how the system scales horizontally

Remember to include version information and note any significant architectural differences between major versions, as indicated by the version history in the README.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Style definitions
    classDef client fill:#90EE90,stroke:#333,stroke-width:2px
    classDef server fill:#87CEEB,stroke:#333,stroke-width:2px
    classDef storage fill:#DEB887,stroke:#333,stroke-width:2px
    classDef monitoring fill:#FFD700,stroke:#333,stroke-width:2px
    classDef core fill:#B19CD9,stroke:#333,stroke-width:2px
    classDef transaction fill:#FF9999,stroke:#333,stroke-width:2px

    %% Client Layer
    subgraph ClientLayer
        C_API["C Binding"]:::client
        Java["Java Binding"]:::client
        Python["Python Binding"]:::client
        Go["Go Binding"]:::client
        Ruby["Ruby Binding"]:::client
    end

    %% Server Components
    subgraph ServerComponents
        CC["Cluster Controller"]:::server
        CP["Commit Proxy"]:::server
        SS["Storage Server"]:::server
        TLog["Transaction Log"]:::server
        Master["Master"]:::server
        Resolver["Resolver"]:::server
        Coord["Coordinators"]:::server
    end

    %% Storage Layer
    subgraph StorageLayer
        KVStore["Key-Value Store"]:::storage
        RocksDB["RocksDB Backend"]:::storage
        StorageMetrics["Storage Metrics"]:::storage
        DataDist["Data Distribution"]:::storage
    end

    %% Transaction System
    subgraph TransactionSystem
        DbContext["Database Context"]:::transaction
        RYW["Read Your Writes"]:::transaction
        CommitTx["Commit Transaction"]:::transaction
        ConflictSet["Conflict Detection"]:::transaction
    end

    %% Monitoring & Management
    subgraph MonitoringSystem
        Status["Status Reporter"]:::monitoring
        MetricLogger["Metric Logger"]:::monitoring
        BackupSystem["Backup System"]:::monitoring
        BackupAgent["Backup Agent"]:::monitoring
    end

    %% Core Infrastructure
    subgraph CoreInfra
        ActorFramework["Actor Framework"]:::core
        NetworkLayer["Network Layer"]:::core
        CoreClient["Core Client"]:::core
    end

    %% Relationships
    ClientLayer --> CP
    ClientLayer --> SS
    CP --> Resolver
    CP --> TLog
    TLog --> SS
    CC --> Master
    Master --> CP
    SS --> StorageLayer
    CP --> TransactionSystem
    MonitoringSystem --> ServerComponents

    %% Click events
    click C_API "https://github.com/apple/foundationdb/tree/main/bindings/c/"
    click Java "https://github.com/apple/foundationdb/tree/main/bindings/java/"
    click Python "https://github.com/apple/foundationdb/tree/main/bindings/python/"
    click Go "https://github.com/apple/foundationdb/tree/main/bindings/go/"
    click Ruby "https://github.com/apple/foundationdb/tree/main/bindings/ruby/"
    click CC "https://github.com/apple/foundationdb/blob/main/fdbserver/ClusterController.actor.cpp"
    click CP "https://github.com/apple/foundationdb/blob/main/fdbserver/CommitProxyServer.actor.cpp"
    click SS "https://github.com/apple/foundationdb/blob/main/fdbserver/storageserver.actor.cpp"
    click TLog "https://github.com/apple/foundationdb/blob/main/fdbserver/TLogServer.actor.cpp"
    click Master "https://github.com/apple/foundationdb/blob/main/fdbserver/masterserver.actor.cpp"
    click Resolver "https://github.com/apple/foundationdb/blob/main/fdbserver/Resolver.actor.cpp"
    click Coord "https://github.com/apple/foundationdb/blob/main/fdbserver/Coordination.actor.cpp"
    click KVStore "https://github.com/apple/foundationdb/blob/main/fdbserver/KeyValueStoreMemory.actor.cpp"
    click RocksDB "https://github.com/apple/foundationdb/blob/main/fdbserver/KeyValueStoreRocksDB.actor.cpp"
    click StorageMetrics "https://github.com/apple/foundationdb/blob/main/fdbserver/StorageMetrics.actor.cpp"
    click DataDist "https://github.com/apple/foundationdb/blob/main/fdbserver/DataDistribution.actor.cpp"
    click DbContext "https://github.com/apple/foundationdb/blob/main/fdbclient/DatabaseContext.cpp"
    click RYW "https://github.com/apple/foundationdb/blob/main/fdbclient/ReadYourWrites.actor.cpp"
    click CommitTx "https://github.com/apple/foundationdb/blob/main/fdbclient/CommitTransaction.h"
    click ConflictSet "https://github.com/apple/foundationdb/blob/main/fdbserver/ConflictSet.h"
    click Status "https://github.com/apple/foundationdb/blob/main/fdbserver/Status.actor.cpp"
    click MetricLogger "https://github.com/apple/foundationdb/blob/main/fdbserver/MetricLogger.actor.cpp"
    click BackupSystem "https://github.com/apple/foundationdb/tree/main/fdbbackup/"
    click BackupAgent "https://github.com/apple/foundationdb/blob/main/fdbclient/BackupAgent.actor.h"
    click ActorFramework "https://github.com/apple/foundationdb/tree/main/flow/"
    click NetworkLayer "https://github.com/apple/foundationdb/tree/main/fdbrpc/"
    click CoreClient "https://github.com/apple/foundationdb/tree/main/fdbclient/"

    %% Legend
    subgraph Legend
        Client["Client Components"]:::client
        ServerComp["Server Components"]:::server
        StorageComp["Storage Components"]:::storage
        MonitoringComp["Monitoring Components"]:::monitoring
        CoreComp["Core Infrastructure"]:::core
        TxComp["Transaction Components"]:::transaction
    end
```
