How himjoe/dgraph works
Project type and purpose
• This is Dgraph: a horizontally-scalable, distributed GraphQL database (no UI or “frontend” code). It provides ACID transactions, consistent replication, and native GraphQL query endpoints over HTTP/gRPC.
High-level architecture components
a. Dgraph Zero
• Coordinates cluster membership, assigns unique IDs, manages RAFT groups.
• Code in dgraph/cmd/zero, plus Raft WAL in raftwal/.
b. Dgraph Alpha
• Stores graph data (shards), serves queries/mutations.
• Exposes HTTP/gRPC + JSON/Protobuf.
• Core modules:
– Storage: posting lists (posting/), RocksDB-backed file store (filestore/).
– Consensus: RAFT replication via raftwal/, conn/raft_server.go.
– Query engine: planner & executor (query/, algo/).
– GraphQL layer: server (edgraph/), schema & resolvers (graphql/schema, graphql/resolve).
– Background workers: backup, restore, change data capture (worker/).
c. Clients & CLI tools
• dgraph CLI (commands: alpha, zero, bulk loader, live loader, admin).
• dgraphapi client helpers.
d. External services
• Object storage for backups (MinIO/S3 via worker/, testutil/minio.go, x/minioclient.go).
• Monitoring/metrics (Prometheus config under compose/).
• Tracing/Telemetry (ocagent in ocagent/).
Component interactions and data flow
• Clients (GraphQL/HTTP or gRPC) → Alpha nodes (load-balanced).
• Alpha ↔ Zero (gRPC) for heartbeats, membership, shard assignments.
• Alpha nodes form RAFT groups (via raftwal/), replicate data among themselves.
• Alpha writes to posting lists on disk (posting/), and snapshots via worker/.
• Query requests traverse: HTTP layer → edgraph/ GraphQL server → query/ planner → posting/ storage → result.
• Mutations go: HTTP → edgraph/ → worker/mutation.go → RAFT log → posting lists updated.
• Backup/restore operations: scheduled by user CLI → worker/backup.go + object store client.
• Metrics emitted to Prometheus endpoints (in conn/metrics.go, x/metrics.go).
Important architectural patterns
• Modular layering: CLI → server → GraphQL layer → query planner → storage.
• Microservice-style separation: Zero vs. Alpha as independent processes.
• Event-driven, append-only RAFT log for consensus (in raftwal/).
• Pluggable storage abstraction: local files vs. remote via filestore/.
• GraphQL schema generation & resolvers auto-generated (graphql/schema, graphql/resolve).