How ethersphere/bee works
Project type and purpose
• This is “Swarm Bee,” a Go‑based, peer‑to‑peer node implementation for the Ethereum Swarm storage network. It provides:
• A CLI (cmd/bee)
• A JSON/HTTP API server (defined under pkg/api)
• Core P2P networking (libp2p under pkg/p2p/libp2p)
• Chunk storage, retrieval, and synchronization modules
• Postage stamp issuance and on‑chain settlement integration
• Metrics, health checks, access control, and other supporting services
File structure
• Top‑level components:
• cmd/bee: entry‑point and subcommands (init, start, deploy, version…)
• pkg/api: HTTP handlers, OpenAPI spec under openapi/
• pkg/p2p: libp2p host, discovery, stream protocols, mock implementations
• pkg/storage + pkg/statestore + pkg/chunkstore: persistent stores (leveldb, in‑memory)
• Modules for services: pkg/postage, pkg/retrieval, pkg/pullsync, pkg/pushsync, pkg/accounting, pkg/settlement, pkg/accesscontrol, etc.
• Utilities: pkg/log, pkg/metrics, pkg/discovery, pkg/config, pkg/crypto
• Tests are co‑located with packages, signifying modular responsibilities
• Config and packaging files for Docker, CI, Homebrew, RPM, DEB
README for additional insights
• It describes the node’s purpose: run a Swarm Bee node, expose a REST API, participate in P2P.
• Versioning notes emphasize that P2P wire protocols live inside Bee, which can break on minor version bumps.
• OpenAPI spec links point to the API contract, which is served by pkg/api.
a. Main components:
• CLI layer (cmd/bee)
• Configuration manager (pkg/config)
• API server (HTTP router in pkg/api)
• Core node manager (pkg/node): initializes subsystems
• P2P networking (libp2p host in pkg/p2p/libp2p, discovery in pkg/discovery)
• Storage stack:
• Chunk storage (pkg/storage, pkg/statestore)
• State store (under pin, KV, leveldb)
• Synchronization services:
• PullSync (pkg/pullsync)
• PushSync (pkg/pushsync)
• Retrieval service (pkg/retrieval)
• Postage/Stamping service (pkg/postage)
• Settlement and accounting:
• Chequebook (under pkg/accounting and pkg/settlement/swap)
• Supporting services: Access control (pkg/accesscontrol), logging (pkg/log), metrics (pkg/metrics), health/probes, CORS
• External services:
• P2P peers over libp2p
• Ethereum JSON‑RPC node (for chainstate and postage contract interactions)
• Clients consuming the HTTP API
b. Relationships and interactions:
• The CLI invokes core initialization, passes configuration to all subsystems.
• Core node manager spins up the P2P host, storage backends, and service modules.
• API server routes requests to the respective service modules (e.g., GET /chunks → storage, POST /stamps → postage).
• P2P protocols use discovery to find peers and pull/push sync to exchange chunks.
• Postage service listens for on‑chain events (Ethereum RPC) to issue stamps.
• Settlement module periodically issues cheques and invoices peers, interacting with the Ethereum RPC.
• Storage and statestore persist chunks and metadata, which retrieval and sync services consume.
c. Architectural patterns and design principles:
• Modular service layers: each feature (sync, retrieval, postage) lives in its own package and interface.
• Dependency injection/mocks: most packages define interfaces and ship mocks for testing.
• Event‑driven on‑chain listeners: services subscribe to chain events (postage, chequebook).
• Layered architecture: CLI → Core → Service modules → Storage/P2P → External.
d. Key technologies/frameworks/libraries:
• Go modules (go.mod)
• libp2p for P2P networking (pkg/p2p/libp2p)
• LevelDB for local storage (pkg/storage/leveldbstore, pkg/statestore/leveldb)
• HTTP router and OpenAPI spec (pkg/api, openapi/Swarm.yaml)
• CI/CD: GitHub Actions workflows, goreleaser
• Docker packaging and multi‑arch images
Elements
• Clear labels for each component (rectangle for each service layer)
• Configuration flow (CLI → Core)
• API request flow (Client → API → Modules → Storage/P2P)
• Data exchange flow (P2P host ↔ Peer)
• Event flow (Ethereum RPC → Postage/Settlement → Core)
• Blue for network components (P2P, discovery)
• Green for storage components
• Orange for service modules (postage, retrieval, sync)
• Grey for external systems (Peers, Ethereum node, Clients)
Detailed capture of essential elements
• Do not combine services; show each package’s responsibility
• Interfaces where they abstract implementation (e.g., storage, statestore, p2p)
• Mocks and testing stubs only if illustrating testability or DI—but optional