# juicedata/juicefs architecture

> Architecture diagram of the GitHub repository juicedata/juicefs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/juicedata/juicefs
- Repository: https://github.com/juicedata/juicefs

## Overview

#### Purpose and executable entry points
JuiceFS is a Go-based, cloud-native distributed POSIX filesystem client. It separates filesystem metadata from file data: metadata resides in a transactional database and chunked data in object or filesystem-backed storage. The root executable delegates to the CLI command layer: `main.go`, `cmd/main.go`, and command implementations such as `cmd/mount.go`, `cmd/format.go`, and `cmd/gateway.go`.

#### Client composition and mount lifecycle
The `mount` command constructs the storage stack, virtual filesystem, and OS-specific mount adapter. `pkg/vfs` implements filesystem semantics over metadata and chunk storage; `pkg/fuse` exposes it to Linux/macOS through FUSE, while `pkg/winfsp` supplies the Windows integration. Primary paths: `cmd/mount.go`, `pkg/vfs/vfs.go`, `pkg/fuse/fuse.go`, `pkg/winfsp/winfs.go`.

#### Metadata control plane
`pkg/meta` defines the metadata abstraction and filesystem state: namespaces, inode/file attributes, directory entries, slice layouts, locks, quotas, status, backups, and dump/load. Concrete backends span Redis, SQL databases, and transactional KV engines, making this package the consistency and coordination boundary between clients. Paths: `pkg/meta/interface.go`, `pkg/meta/redis.go`, `pkg/meta/sql.go`, `pkg/meta/tkv.go`.

#### Data plane: file layout, caching, and object persistence
File writes are represented as chunk/slice metadata and materialized as fixed-size blocks in the configured object store. `pkg/chunk` coordinates block reads/writes, memory/disk caches, prefetching, eviction, and deduplication of concurrent fetches. The VFS binds metadata operations to this chunk store. Paths: `pkg/vfs/writer.go`, `pkg/vfs/reader.go`, `pkg/chunk/cached_store.go`, `pkg/chunk/cache_manager.go`.

#### Storage-provider boundary
`pkg/object` provides the common object-storage interface plus adapters for S3-compatible services, public clouds, HDFS, local filesystems, Redis, SQL, and network filesystems. Wrappers add encryption, prefixes, sharding, checksums, and REST behavior without coupling VFS/chunk logic to a provider. Paths: `pkg/object/interface.go`, `pkg/object/object_storage.go`, `pkg/object/s3.go`, `pkg/object/encrypt.go`.

#### Alternative access interfaces and integrations
Beyond POSIX mounts, the client offers S3 gateway and WebDAV services, plus a Hadoop-compatible Java filesystem SDK. The Java implementation bridges to Go-native filesystem functionality and integrates with Hadoop/Flink ecosystems; Python packaging exposes another client-facing SDK surface. Paths: `pkg/gateway/gateway.go`, `cmd/webdav.go`, `sdk/java/src/main/java/io/juicefs/JuiceFileSystemImpl.java`, `sdk/python/juicefs/juicefs/juicefs.py`.

#### Operations, observability, and deployment boundaries
CLI operational commands format/destroy filesystems, inspect or repair metadata, manage garbage collection, compact data, clone/restore, synchronize, and report status. Metrics are emitted from filesystem/chunk layers; optional anonymous usage reporting is isolated. Runtime targets include Linux, macOS, and Windows; deployments commonly connect to external metadata engines (Redis/MySQL/PostgreSQL/SQLite/TiKV/etcd/FoundationDB) and object stores, with Kubernetes integration documented/deployed separately. Paths: `cmd/fsck.go`, `cmd/gc.go`, `cmd/sync.go`, `pkg/metric/metrics.go`, `pkg/usage/usage.go`.

Main entry points: [`pkg/vfs/vfs.go`](https://github.com/juicedata/juicefs/blob/main/pkg/vfs/vfs.go) (Virtual filesystem), [`pkg/chunk/cached_store.go`](https://github.com/juicedata/juicefs/blob/main/pkg/chunk/cached_store.go) (Chunk store), [`cmd/mount.go`](https://github.com/juicedata/juicefs/blob/main/cmd/mount.go) (Mount command), [`cmd/main.go`](https://github.com/juicedata/juicefs/blob/main/cmd/main.go) (CLI commands)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Commands"]
  node_main["Root executable<br/>Go entry point<br/>[main.go]"]
  node_cli["CLI commands<br/>command layer<br/>[main.go]"]
  node_mount["Mount command<br/>mount lifecycle<br/>[mount.go]"]
  node_operations["Operations commands<br/>admin workflows<br/>[fsck.go]"]
end

subgraph group_core["Filesystem Core"]
  node_vfs["Virtual filesystem<br/>POSIX semantics<br/>[vfs.go]"]
  node_io["VFS readers and writers<br/>file I/O layer<br/>[writer.go]"]
  node_metadata[("Metadata abstraction<br/>control plane<br/>[interface.go]")]
  node_chunk["Chunk store<br/>data plane<br/>[cached_store.go]"]
  node_cache["Cache manager<br/>memory/disk cache<br/>[cache_manager.go]"]
  node_metrics["Metrics<br/>observability<br/>[metrics.go]"]
  node_usage["Usage reporting<br/>optional telemetry<br/>[usage.go]"]
end

subgraph group_access["Access Interfaces"]
  node_fuse["FUSE adapter<br/>Linux/macOS mount<br/>[fuse.go]"]
  node_winfsp["WinFSP adapter<br/>Windows mount<br/>[winfs.go]"]
  node_gateway["S3 gateway<br/>S3 service<br/>[gateway.go]"]
  node_webdav["WebDAV command<br/>WebDAV service<br/>[webdav.go]"]
  node_java_sdk["Hadoop Java SDK<br/>Java filesystem SDK"]
  node_python_sdk["Python SDK<br/>client SDK<br/>[juicefs.py]"]
end

subgraph group_storage["Storage Providers"]
  node_metadata_backends[("Metadata backends<br/>[sql.go]")]
  node_object["Object interface and adapters<br/>provider boundary<br/>[interface.go]"]
  node_object_wrappers["Object wrappers<br/>storage transforms<br/>[encrypt.go]"]
  node_providers[("External object stores<br/>data persistence")]
end

node_main -->|"delegates to"| node_cli
node_cli -->|"runs"| node_mount
node_cli -->|"runs"| node_operations
node_mount -->|"constructs"| node_vfs
node_mount -->|"mounts on Linux/macOS"| node_fuse
node_mount -->|"mounts on Windows"| node_winfsp
node_fuse -->|"exposes"| node_vfs
node_winfsp -->|"exposes"| node_vfs
node_vfs -->|"stores filesystem state"| node_metadata
node_vfs -->|"performs file I/O"| node_io
node_io -->|"reads and writes blocks"| node_chunk
node_metadata -->|"implemented by"| node_metadata_backends
node_chunk -->|"manages"| node_cache
node_chunk -->|"persists blocks via"| node_object
node_object -->|"composed with"| node_object_wrappers
node_object_wrappers -->|"targets"| node_providers
node_gateway -->|"serves filesystem through"| node_vfs
node_webdav -->|"serves filesystem through"| node_vfs
node_java_sdk -.->|"bridges to"| node_vfs
node_python_sdk -.->|"client surface for"| node_vfs
node_vfs -.->|"emits"| node_metrics
node_chunk -.->|"emits"| node_metrics
node_vfs -.->|"optionally reports"| node_usage
node_operations -->|"inspects and repairs"| node_metadata
node_operations -->|"GC, compacts, and syncs"| node_chunk

click node_main "https://github.com/juicedata/juicefs/blob/main/main.go"
click node_cli "https://github.com/juicedata/juicefs/blob/main/cmd/main.go"
click node_mount "https://github.com/juicedata/juicefs/blob/main/cmd/mount.go"
click node_operations "https://github.com/juicedata/juicefs/blob/main/cmd/fsck.go"
click node_vfs "https://github.com/juicedata/juicefs/blob/main/pkg/vfs/vfs.go"
click node_io "https://github.com/juicedata/juicefs/blob/main/pkg/vfs/writer.go"
click node_metadata "https://github.com/juicedata/juicefs/blob/main/pkg/meta/interface.go"
click node_metadata_backends "https://github.com/juicedata/juicefs/blob/main/pkg/meta/sql.go"
click node_chunk "https://github.com/juicedata/juicefs/blob/main/pkg/chunk/cached_store.go"
click node_cache "https://github.com/juicedata/juicefs/blob/main/pkg/chunk/cache_manager.go"
click node_object "https://github.com/juicedata/juicefs/blob/main/pkg/object/interface.go"
click node_object_wrappers "https://github.com/juicedata/juicefs/blob/main/pkg/object/encrypt.go"
click node_fuse "https://github.com/juicedata/juicefs/blob/main/pkg/fuse/fuse.go"
click node_winfsp "https://github.com/juicedata/juicefs/blob/main/pkg/winfsp/winfs.go"
click node_gateway "https://github.com/juicedata/juicefs/blob/main/pkg/gateway/gateway.go"
click node_webdav "https://github.com/juicedata/juicefs/blob/main/cmd/webdav.go"
click node_java_sdk "https://github.com/juicedata/juicefs/blob/main/sdk/java/src/main/java/io/juicefs/JuiceFileSystemImpl.java"
click node_python_sdk "https://github.com/juicedata/juicefs/blob/main/sdk/python/juicefs/juicefs/juicefs.py"
click node_metrics "https://github.com/juicedata/juicefs/blob/main/pkg/metric/metrics.go"
click node_usage "https://github.com/juicedata/juicefs/blob/main/pkg/usage/usage.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_cli,node_mount,node_operations toneBlue
class node_vfs,node_io,node_metadata,node_chunk,node_cache,node_metrics,node_usage toneAmber
class node_fuse,node_winfsp,node_gateway,node_webdav,node_java_sdk,node_python_sdk toneMint
class node_metadata_backends,node_object,node_object_wrappers,node_providers toneRose
```

## Components

### Commands

Executable and operational command layer.

- [Root executable](https://github.com/juicedata/juicefs/blob/main/main.go): Go entry point · `main.go`
- [CLI commands](https://github.com/juicedata/juicefs/blob/main/cmd/main.go): command layer · `cmd/main.go`
- [Mount command](https://github.com/juicedata/juicefs/blob/main/cmd/mount.go): mount lifecycle · `cmd/mount.go`
- [Operations commands](https://github.com/juicedata/juicefs/blob/main/cmd/fsck.go): admin workflows · Repair, garbage collection, synchronization, and filesystem administration. · `cmd/fsck.go`

### Filesystem Core

Metadata, VFS, and chunk data-path layers.

- [Virtual filesystem](https://github.com/juicedata/juicefs/blob/main/pkg/vfs/vfs.go): POSIX semantics · Binds metadata state to file-data operations. · `pkg/vfs/vfs.go`
- [VFS readers and writers](https://github.com/juicedata/juicefs/blob/main/pkg/vfs/writer.go): file I/O layer · Translates file reads and writes into chunk and slice operations. · `pkg/vfs/writer.go`
- [Metadata abstraction](https://github.com/juicedata/juicefs/blob/main/pkg/meta/interface.go): control plane · Transactional namespace, inode, slice, lock, quota, and lifecycle state. · `pkg/meta/interface.go`
- [Chunk store](https://github.com/juicedata/juicefs/blob/main/pkg/chunk/cached_store.go): data plane · Coordinates fixed-size block reads, writes, caching, prefetch, and deduplicated fetches. · `pkg/chunk/cached_store.go`
- [Cache manager](https://github.com/juicedata/juicefs/blob/main/pkg/chunk/cache_manager.go): memory/disk cache · `pkg/chunk/cache_manager.go`
- [Metrics](https://github.com/juicedata/juicefs/blob/main/pkg/metric/metrics.go): observability · `pkg/metric/metrics.go`
- [Usage reporting](https://github.com/juicedata/juicefs/blob/main/pkg/usage/usage.go): optional telemetry · Isolated anonymous usage reporting. · `pkg/usage/usage.go`

### Access Interfaces

Mount and protocol-facing adapters.

- [FUSE adapter](https://github.com/juicedata/juicefs/blob/main/pkg/fuse/fuse.go): Linux/macOS mount · `pkg/fuse/fuse.go`
- [WinFSP adapter](https://github.com/juicedata/juicefs/blob/main/pkg/winfsp/winfs.go): Windows mount · `pkg/winfsp/winfs.go`
- [S3 gateway](https://github.com/juicedata/juicefs/blob/main/pkg/gateway/gateway.go): S3 service · `pkg/gateway/gateway.go`
- [WebDAV command](https://github.com/juicedata/juicefs/blob/main/cmd/webdav.go): WebDAV service · `cmd/webdav.go`
- [Hadoop Java SDK](https://github.com/juicedata/juicefs/blob/main/sdk/java/src/main/java/io/juicefs/JuiceFileSystemImpl.java): Java filesystem SDK · Bridges Go-native filesystem capabilities to Hadoop and Flink. · `sdk/java/src/main/java/io/juicefs/JuiceFileSystemImpl.java`
- [Python SDK](https://github.com/juicedata/juicefs/blob/main/sdk/python/juicefs/juicefs/juicefs.py): client SDK · `sdk/python/juicefs/juicefs/juicefs.py`

### Storage Providers

Provider-neutral persistence boundaries.

- [Metadata backends](https://github.com/juicedata/juicefs/blob/main/pkg/meta/sql.go): Redis / SQL / KV · `pkg/meta/sql.go`
- [Object interface and adapters](https://github.com/juicedata/juicefs/blob/main/pkg/object/interface.go): provider boundary · Uniform object API with cloud, filesystem, database, and network adapters. · `pkg/object/interface.go`
- [Object wrappers](https://github.com/juicedata/juicefs/blob/main/pkg/object/encrypt.go): storage transforms · Applies encryption, prefixes, sharding, checksums, and REST behavior. · `pkg/object/encrypt.go`
- **External object stores**: data persistence

## Connections

- Root executable → CLI commands: delegates to
- CLI commands → Mount command: runs
- CLI commands → Operations commands: runs
- Mount command → Virtual filesystem: constructs
- Mount command → FUSE adapter: mounts on Linux/macOS
- Mount command → WinFSP adapter: mounts on Windows
- FUSE adapter → Virtual filesystem: exposes
- WinFSP adapter → Virtual filesystem: exposes
- Virtual filesystem → Metadata abstraction: stores filesystem state
- Virtual filesystem → VFS readers and writers: performs file I/O
- VFS readers and writers → Chunk store: reads and writes blocks
- Metadata abstraction → Metadata backends: implemented by
- Chunk store → Cache manager: manages
- Chunk store → Object interface and adapters: persists blocks via
- Object interface and adapters → Object wrappers: composed with
- Object wrappers → External object stores: targets
- S3 gateway → Virtual filesystem: serves filesystem through
- WebDAV command → Virtual filesystem: serves filesystem through
- Hadoop Java SDK → Virtual filesystem: bridges to
- Python SDK → Virtual filesystem: client surface for
- Virtual filesystem → Metrics: emits
- Chunk store → Metrics: emits
- Virtual filesystem → Usage reporting: optionally reports
- Operations commands → Metadata abstraction: inspects and repairs
- Operations commands → Chunk store: GC, compacts, and syncs
