How anyproto/any-sync works
Project Type & Purpose
• This is a backend “sync node” service implementing the Any-Sync protocol in Go.
• It exposes RPC endpoints (via dRPC) for clients and peers, manages spaces (CRDT-based graph databases), files, ACLs, consensus, coordinator, nameservice, identity, and payments.
• It is a modular monolith broken into service components and network/storage layers.
Core Layers & Directories
A. “Edge” / Clients Layer
• Anytype UI or other local applications (SDKs)
• Peer Sync Peers (other Any-Sync Nodes)
B. RPC & Network Layer (net/)
• net/transport (QUIC, Yamux) for raw multiplexed streams
• net/streampool & net/pool for connection & stream management
• net/rpc (drpc server & client) for RPC dispatch
• net/secureservice for handshake & E2E encryption
• net/peerservice for peer discovery & connection pooling
C. Service & Protocol Layer
• commonspace/spaceservice: space CRDT orchestration (head sync, object sync, tree sync, deletion, settings)
• commonfile/fileservice: file block storage & file sync
• accountservice: account setup & authorization
• acl: access control list service
• consensus/consensusclient: watch ACL changes via consensus node
• coordinator/coordinatorclient: retrieve network configuration from coordinator node
• nameservice/nameserviceclient: DNS-like discovery
• identityrepo: identity management proto
• paymentservice: quota & payment tiers
• metric: metrics collection & reporting
D. Core Engine & Utilities
• commonspace/* submodules (object tree builder, object manager, peer manager, deletion manager)
• app/ (ldiff vs olddiff, logger, ocache, debugstat) for local caching, logging, and diffing
• util/ (crypto, syncqueues, multiqueue, periodicsync, storeutil)
E. Storage Layer
• commonspace/spacestorage (migration, oldstorage, new storage)
• commonfile/fileblockstore
• nodeconf/nodeconfstore for node configuration persistence
• util/crypto keystorage
Data Flows & Interactions
• Clients → dRPC Server (net/rpc) → Service Handlers → Core Engine → Storage → back up the same chain.
• Peer Sync: Service Handlers invoke commonspace/sync → net/secureservice → net/transport → Peer → reverse path.
• External Dependencies: services call coordinatorclient, consensusclient, nameserviceclient via dRPC or DRPC-proto stubs.
Architectural Patterns & Design Principles
• Layered architecture with clear separation (Network → Service → Engine → Storage).
• Modular services, each with its own proto definitions and client stubs.
• Dependency injection & mocking heavy for testability (see mock_* packages everywhere).
• CRDT pipelines: head sync → object sync → tree sync, each as a separate flow.
• Use of multiplexed connections and streams (Yamux/QUIC) under the hood.
Technologies & Frameworks
• Go modules and package structure.
• Protocols: Protocol Buffers & dRPC.
• Transport: QUIC, Yamux, TCP.
• Crypto: ed25519, x25519, AES, user-controlled keys.