# himjoe/dgraph architecture

> Architecture diagram of the GitHub repository himjoe/dgraph, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/himjoe/dgraph
- Repository: https://github.com/himjoe/dgraph

## Overview

##### Project type and purpose
- This is Dgraph: a horizontally-scalable, distributed GraphQL database (no UI or “frontend” code). It provides ACID transactions, consistent replication, and native GraphQL query endpoints over HTTP/gRPC.

##### High-level architecture components
a. Dgraph Zero
- Coordinates cluster membership, assigns unique IDs, manages RAFT groups.
- Code in `dgraph/cmd/zero`, plus Raft WAL in `raftwal/`.
b. Dgraph Alpha
- Stores graph data (shards), serves queries/mutations.
- Exposes HTTP/gRPC + JSON/Protobuf.
- Core modules:
– Storage: posting lists (`posting/`), RocksDB-backed file store (`filestore/`).
– Consensus: RAFT replication via `raftwal/`, `conn/raft_server.go`.
– Query engine: planner & executor (`query/`, `algo/`).
– GraphQL layer: server (`edgraph/`), schema & resolvers (`graphql/schema`, `graphql/resolve`).
– Background workers: backup, restore, change data capture (`worker/`).
c. Clients & CLI tools
- `dgraph` CLI (commands: alpha, zero, bulk loader, live loader, admin).
- `dgraphapi` client helpers.
d. External services
- Object storage for backups (MinIO/S3 via `worker/`, `testutil/minio.go`, `x/minioclient.go`).
- Monitoring/metrics (Prometheus config under `compose/`).
- Tracing/Telemetry (ocagent in `ocagent/`).

##### Component interactions and data flow
- Clients (GraphQL/HTTP or gRPC) → Alpha nodes (load-balanced).
- Alpha ↔ Zero (gRPC) for heartbeats, membership, shard assignments.
- Alpha nodes form RAFT groups (via `raftwal/`), replicate data among themselves.
- Alpha writes to posting lists on disk (`posting/`), and snapshots via `worker/`.
- Query requests traverse: HTTP layer → `edgraph/` GraphQL server → `query/` planner → `posting/` storage → result.
- Mutations go: HTTP → `edgraph/` → `worker/mutation.go` → RAFT log → posting lists updated.
- Backup/restore operations: scheduled by user CLI → `worker/backup.go` + object store client.
- Metrics emitted to Prometheus endpoints (in `conn/metrics.go`, `x/metrics.go`).

##### Important architectural patterns
- Modular layering: CLI → server → GraphQL layer → query planner → storage.
- Microservice-style separation: Zero vs. Alpha as independent processes.
- Event-driven, append-only RAFT log for consensus (in `raftwal/`).
- Pluggable storage abstraction: local files vs. remote via `filestore/`.
- GraphQL schema generation & resolvers auto-generated (`graphql/schema`, `graphql/resolve`).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Clients & CLI
    subgraph "Clients & CLI Tools"
        Clients["Clients (GraphQL HTTP/gRPC)"]:::client
        CLI["CLI Tools"]:::client
        DgraphAPI["Go Client Library"]:::client
    end

    %% Control Plane
    subgraph "Zero Cluster (Control Plane)"
        Zero["Zero Node"]:::core
        RaftWAL["RAFT WAL"]:::core
    end

    %% Data Plane
    subgraph "Alpha Cluster (Data Plane)"
        Alpha["Alpha Node"]:::core
        subgraph "Alpha Components"
            HTTP["HTTP/gRPC API"]:::core
            GraphQLSrv["GraphQL Server"]:::core
            Schema["Schema & Resolvers"]:::core
            Query["Query Planning & Execution Engine"]:::core
            Posting["Posting List Storage"]:::storage
            FileStore["RocksDB File Store"]:::storage
            RaftModule["RAFT Consensus Module"]:::core
            Worker["Background Workers"]:::core
        end
    end

    %% External Services
    subgraph "External Services"
        MinIO["Object Storage (S3/MinIO)"]:::external
        Prom["Prometheus Monitoring"]:::external
        OC["OpenCensus Agent"]:::external
    end

    %% Connections
    Clients -->|"HTTP/gRPC"| HTTP
    CLI -->|management RPC| Zero
    CLI -->|data load| Alpha
    DgraphAPI -->|"gRPC/HTTP"| HTTP

    HTTP -->|calls| GraphQLSrv
    GraphQLSrv -->|translates| Query
    Query -->|reads/writes| Posting
    Query -->|reads/writes| FileStore
    HTTP -->|mutations| Worker
    Worker -->|RAFT log entry| RaftModule
    RaftModule -->|append-only log| RaftWAL
    Alpha -->|"membership RPC (gRPC)"| Zero
    RaftModule -->|RAFT replication| Alpha
    Worker -->|"S3 API"| MinIO
    Alpha -->|"metrics scrape"| Prom
    Alpha -->|"tracing"| OC

    %% Click Events
    click Zero "https://github.com/himjoe/dgraph/tree/main/dgraph/cmd/zero"
    click RaftWAL "https://github.com/himjoe/dgraph/tree/main/raftwal/"
    click Alpha "https://github.com/himjoe/dgraph/tree/main/dgraph/cmd/alpha"
    click Posting "https://github.com/himjoe/dgraph/tree/main/posting/"
    click FileStore "https://github.com/himjoe/dgraph/tree/main/filestore/"
    click RaftModule "https://github.com/himjoe/dgraph/tree/main/raftwal/"
    click RaftModule "https://github.com/himjoe/dgraph/blob/main/conn/raft_server.go"
    click Query "https://github.com/himjoe/dgraph/tree/main/query/"
    click Query "https://github.com/himjoe/dgraph/tree/main/algo/"
    click GraphQLSrv "https://github.com/himjoe/dgraph/tree/main/edgraph/"
    click GraphQLSrv "https://github.com/himjoe/dgraph/blob/main/edgraph/server.go"
    click GraphQLSrv "https://github.com/himjoe/dgraph/blob/main/edgraph/graphql.go"
    click Schema "https://github.com/himjoe/dgraph/tree/main/graphql/schema/"
    click Schema "https://github.com/himjoe/dgraph/tree/main/graphql/resolve/"
    click Worker "https://github.com/himjoe/dgraph/tree/main/worker/"
    click Worker "https://github.com/himjoe/dgraph/blob/main/worker/backup.go"
    click Worker "https://github.com/himjoe/dgraph/blob/main/worker/restore_map.go"
    click Worker "https://github.com/himjoe/dgraph/blob/main/worker/cdc.go"
    click Worker "https://github.com/himjoe/dgraph/blob/main/worker/mutation.go"
    click CLI "https://github.com/himjoe/dgraph/tree/main/dgraph/cmd/"
    click DgraphAPI "https://github.com/himjoe/dgraph/tree/main/dgraphapi/"
    click MinIO "https://github.com/himjoe/dgraph/blob/main/x/minioclient.go"
    click MinIO "https://github.com/himjoe/dgraph/blob/main/testutil/minio.go"
    click Prom "https://github.com/himjoe/dgraph/blob/main/compose/prometheus.yml"
    click OC "https://github.com/himjoe/dgraph/tree/main/ocagent/"

    %% Styles
    classDef core fill:#cce5ff,stroke:#000,stroke-width:1px
    classDef storage fill:#d4edda,stroke:#000,stroke-width:1px
    classDef external fill:#fff3cd,stroke:#000,stroke-width:1px
    classDef client fill:#e2e3e5,stroke:#000,stroke-width:1px
```
