# apache/opendal architecture

> Architecture diagram of the GitHub repository apache/opendal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/opendal
- Repository: https://github.com/apache/opendal

## Overview

##### Main Components of the System

**┌─────────────────────────────────────────────────────────┐**
- Client Applications                    │
- ┘
- ↓
**┌─────────────────────────────────────────────────────────┐**
- Language Bindings                     │
- Rust │ Java │ Python │ Node.js │ Go │ C/C++ │ Others   │
- ┘
- ↓
**┌─────────────────────────────────────────────────────────┐**
- OpenDAL Core                         │
- ┌─────────────┐  ┌──────────┐  ┌──────────────┐     │
- Services  │  │  Layers  │  │    Types     │     │
- ┘  └──────────┘  └──────────────┘     │
- ┘
- ↓
**┌─────────────────────────────────────────────────────────┐**
- Storage Services                      │
- ┘

##### Component Details

a. Storage Services Layer (Bottom):
- Services by type:
- Standard Protocols (FTP, HTTP, SFTP, WebDAV)
- Object Storage (S3, Azure Blob, GCS, etc.)
- File Storage (FS, HDFS, etc.)
- Key-Value Storage (Redis, etcd, etc.)
- Database Storage (MongoDB, MySQL, etc.)
- Cache Storage (Memcached, etc.)

b. OpenDAL Core (Middle):
- Services: Implementation of storage service adapters
- Layers: Cross-cutting concerns (retry, logging, metrics)
- Types: Core data structures and traits

c. Language Bindings (Upper Middle):
- Supported language bindings
- Stability status (released vs. in development)

d. Client Applications (Top):
- CLI Tools (oli, oay, ofs)
- Integration Points (FUSE, WebDAV, etc.)

##### Connection and Data Flow

- Data flow direction between layers
- API calls and responses
- Integration points
- Cross-cutting concerns

##### Additional Details

Annotations for:
- Supported operations (read, write, list, delete)
- Capability indicators
- Version compatibility
- Performance characteristics

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles
    classDef core fill:#2374ab,color:white
    classDef binding fill:#48a9a6,color:white
    classDef storage fill:#4b644a,color:white
    classDef cli fill:#c1666b,color:white
    classDef integration fill:#d4b483,color:black

    %% Client Applications Layer
    subgraph ClientApps["Client Applications"]
        CLI["CLI Tools"]
        CLI --> |Contains| oli["oli"]:::cli
        CLI --> |Contains| oay["oay"]:::cli
        CLI --> |Contains| ofs["ofs"]:::cli
        
        Integration["Integration Points"]
        Integration --> |Contains| FUSE["FUSE"]:::integration
        Integration --> |Contains| WebDAVServer["WebDAV Server"]:::integration
    end

    %% Language Bindings Layer
    subgraph Bindings["Language Bindings"]
        Java["Java"]:::binding
        Python["Python"]:::binding
        NodeJS["Node.js"]:::binding
        Go["Go"]:::binding
        C["C"]:::binding
        CPP["C++"]:::binding
        Ruby["Ruby"]:::binding
        Swift["Swift"]:::binding
    end

    %% OpenDAL Core Layer
    subgraph Core["OpenDAL Core"]
        CoreTypes["Core Types"]:::core
        CoreLayers["Core Layers"]:::core
        RawImpl["Raw Implementation"]:::core
    end

    %% Storage Services Layer
    subgraph Storage["Storage Services"]
        subgraph ObjectStorage["Object Storage"]
            S3["S3"]:::storage
            AzBlob["Azure Blob"]:::storage
            GCS["GCS"]:::storage
        end

        subgraph FileStorage["File Storage"]
            FS["FS"]:::storage
            HDFS["HDFS"]:::storage
        end

        subgraph KVStorage["Key-Value Storage"]
            Redis["Redis"]:::storage
            Etcd["Etcd"]:::storage
        end

        subgraph DBStorage["Database Storage"]
            MongoDB["MongoDB"]:::storage
            MySQL["MySQL"]:::storage
        end

        subgraph CacheStorage["Cache Storage"]
            Memcached["Memcached"]:::storage
        end

        subgraph Protocols["Standard Protocols"]
            FTP["FTP"]:::storage
            HTTP["HTTP"]:::storage
            SFTP["SFTP"]:::storage
            WebDAV["WebDAV"]:::storage
        end
    end

    %% Connections between layers
    ClientApps --> Bindings
    Bindings --> Core
    Core --> Storage

    %% Click events for component mapping
    click oli "https://github.com/apache/opendal/tree/main/bin/oli/"
    click oay "https://github.com/apache/opendal/tree/main/bin/oay/"
    click ofs "https://github.com/apache/opendal/tree/main/bin/ofs/"
    
    click Java "https://github.com/apache/opendal/tree/main/bindings/java/"
    click Python "https://github.com/apache/opendal/tree/main/bindings/python/"
    click NodeJS "https://github.com/apache/opendal/tree/main/bindings/nodejs/"
    click Go "https://github.com/apache/opendal/tree/main/bindings/go/"
    click C "https://github.com/apache/opendal/tree/main/bindings/c/"
    click CPP "https://github.com/apache/opendal/tree/main/bindings/cpp/"
    click Ruby "https://github.com/apache/opendal/tree/main/bindings/ruby/"
    click Swift "https://github.com/apache/opendal/tree/main/bindings/swift/"
    
    click CoreTypes "https://github.com/apache/opendal/tree/main/core/src/types/"
    click CoreLayers "https://github.com/apache/opendal/tree/main/core/src/layers/"
    click RawImpl "https://github.com/apache/opendal/tree/main/core/src/raw/"
    
    click S3 "https://github.com/apache/opendal/tree/main/core/src/services/s3/"
    click AzBlob "https://github.com/apache/opendal/tree/main/core/src/services/azblob/"
    click GCS "https://github.com/apache/opendal/tree/main/core/src/services/gcs/"
    click FS "https://github.com/apache/opendal/tree/main/core/src/services/fs/"
    click HDFS "https://github.com/apache/opendal/tree/main/core/src/services/hdfs/"
    click Redis "https://github.com/apache/opendal/tree/main/core/src/services/redis/"
    click Etcd "https://github.com/apache/opendal/tree/main/core/src/services/etcd/"
    click MongoDB "https://github.com/apache/opendal/tree/main/core/src/services/mongodb/"
    click MySQL "https://github.com/apache/opendal/tree/main/core/src/services/mysql/"
    click Memcached "https://github.com/apache/opendal/tree/main/core/src/services/memcached/"
    click FTP "https://github.com/apache/opendal/tree/main/core/src/services/ftp/"
    click HTTP "https://github.com/apache/opendal/tree/main/core/src/services/http/"
    click SFTP "https://github.com/apache/opendal/tree/main/core/src/services/sftp/"
    click WebDAV "https://github.com/apache/opendal/tree/main/core/src/services/webdav/"
    
    click FUSE "https://github.com/apache/opendal/tree/main/integrations/fuse3/"
    click WebDAVServer "https://github.com/apache/opendal/tree/main/integrations/dav-server/"
```
