# ethersphere/bee architecture

> Architecture diagram of the GitHub repository ethersphere/bee, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ethersphere/bee
- Repository: https://github.com/ethersphere/bee

## Overview

##### Project type and purpose
- This is “Swarm Bee,” a Go‑based, peer‑to‑peer node implementation for the Ethereum Swarm storage network. It provides:
- A CLI (`cmd/bee`)
- A JSON/HTTP API server (defined under `pkg/api`)
- Core P2P networking (libp2p under `pkg/p2p/libp2p`)
- Chunk storage, retrieval, and synchronization modules
- Postage stamp issuance and on‑chain settlement integration
- Metrics, health checks, access control, and other supporting services

##### File structure
- Top‑level components:
- `cmd/bee`: entry‑point and subcommands (init, start, deploy, version…)
- `pkg/api`: HTTP handlers, OpenAPI spec under `openapi/`
- `pkg/p2p`: libp2p host, discovery, stream protocols, mock implementations
- `pkg/storage` + `pkg/statestore` + `pkg/chunkstore`: persistent stores (leveldb, in‑memory)
- Modules for services: `pkg/postage`, `pkg/retrieval`, `pkg/pullsync`, `pkg/pushsync`, `pkg/accounting`, `pkg/settlement`, `pkg/accesscontrol`, etc.
- Utilities: `pkg/log`, `pkg/metrics`, `pkg/discovery`, `pkg/config`, `pkg/crypto`
- Tests are co‑located with packages, signifying modular responsibilities
- Config and packaging files for Docker, CI, Homebrew, RPM, DEB

##### README for additional insights
- It describes the node’s purpose: run a Swarm Bee node, expose a REST API, participate in P2P.
- Versioning notes emphasize that P2P wire protocols live inside Bee, which can break on minor version bumps.
- OpenAPI spec links point to the API contract, which is served by `pkg/api`.

a. Main components:
- CLI layer (`cmd/bee`)
- Configuration manager (`pkg/config`)
- API server (HTTP router in `pkg/api`)
- Core node manager (`pkg/node`): initializes subsystems
- P2P networking (libp2p host in `pkg/p2p/libp2p`, discovery in `pkg/discovery`)
- Storage stack:
- Chunk storage (`pkg/storage`, `pkg/statestore`)
- State store (under pin, KV, leveldb)
- Synchronization services:
- PullSync (`pkg/pullsync`)
- PushSync (`pkg/pushsync`)
- Retrieval service (`pkg/retrieval`)
- Postage/Stamping service (`pkg/postage`)
- Settlement and accounting:
- Chequebook (under `pkg/accounting` and `pkg/settlement/swap`)
- Supporting services: Access control (`pkg/accesscontrol`), logging (`pkg/log`), metrics (`pkg/metrics`), health/probes, CORS
- External services:
- P2P peers over libp2p
- Ethereum JSON‑RPC node (for chainstate and postage contract interactions)
- Clients consuming the HTTP API

b. Relationships and interactions:
- The CLI invokes core initialization, passes configuration to all subsystems.
- Core node manager spins up the P2P host, storage backends, and service modules.
- API server routes requests to the respective service modules (e.g., `GET /chunks` → storage, `POST /stamps` → postage).
- P2P protocols use discovery to find peers and pull/push sync to exchange chunks.
- Postage service listens for on‑chain events (Ethereum RPC) to issue stamps.
- Settlement module periodically issues cheques and invoices peers, interacting with the Ethereum RPC.
- Storage and statestore persist chunks and metadata, which retrieval and sync services consume.

c. Architectural patterns and design principles:
- Modular service layers: each feature (sync, retrieval, postage) lives in its own package and interface.
- Dependency injection/mocks: most packages define interfaces and ship mocks for testing.
- Event‑driven on‑chain listeners: services subscribe to chain events (postage, chequebook).
- Layered architecture: CLI → Core → Service modules → Storage/P2P → External.

d. Key technologies/frameworks/libraries:
- Go modules (`go.mod`)
- libp2p for P2P networking (`pkg/p2p/libp2p`)
- LevelDB for local storage (`pkg/storage/leveldbstore`, `pkg/statestore/leveldb`)
- HTTP router and OpenAPI spec (`pkg/api`, `openapi/Swarm.yaml`)
- CI/CD: GitHub Actions workflows, goreleaser
- Docker packaging and multi‑arch images

##### Elements
- Clear labels for each component (rectangle for each service layer)
- Configuration flow (CLI → Core)
- API request flow (Client → API → Modules → Storage/P2P)
- Data exchange flow (P2P host ↔ Peer)
- Event flow (Ethereum RPC → Postage/Settlement → Core)
- Blue for network components (P2P, discovery)
- Green for storage components
- Orange for service modules (postage, retrieval, sync)
- Grey for external systems (Peers, Ethereum node, Clients)

##### Detailed capture of essential elements
- Do not combine services; show each package’s responsibility
- Interfaces where they abstract implementation (e.g., storage, statestore, p2p)
- Mocks and testing stubs only if illustrating testability or DI—but optional

## Diagram (Mermaid)

```mermaid
flowchart TD
    direction TB

    subgraph "CLI Layer"
        CLI["CLI (cmd/bee)"]:::cli
    end

    subgraph "Configuration"
        Config["Config Manager (pkg/config)"]:::config
    end

    subgraph "Core Node"
        Core["Core Node Manager (pkg/node/node.go)"]:::core
    end

    subgraph "API Server"
        API["HTTP API Server (pkg/api)"]:::api
        OpenAPI["OpenAPI Spec (openapi/Swarm.yaml)"]:::api
    end

    subgraph "Service Modules" 
        Pull["PullSync Service (pkg/pullsync/pullsync.go)"]:::service
        Push["PushSync Service (pkg/pushsync/pushsync.go)"]:::service
        Retrieval["Retrieval Service (pkg/retrieval/retrieval.go)"]:::service
        Postage["Postage Service (pkg/postage/service.go)"]:::service
        Listener["On-chain Listener (pkg/postage/listener/listener.go)"]:::service
        Accounting["Accounting (pkg/accounting/accounting.go)"]:::service
        Settlement["Settlement Swap (pkg/settlement/swap)"]:::service
        Pseudo["Pseudo-Settlement (pkg/settlement/pseudosettle)"]:::service
        Access["Access Control (pkg/accesscontrol/controller.go)"]:::service
        Logger["Logging (pkg/log/logger.go)"]:::support
        Metrics["Metrics & Health (pkg/metrics/metrics.go)"]:::support
    end

    subgraph "P2P Networking"
        P2PLC["LibP2P Host & Protocols (pkg/p2p/libp2p)"]:::network
        Discovery["Discovery (pkg/discovery/discovery.go)"]:::network
    end

    subgraph "Storage Stack"
        Storage["Chunk Storage (pkg/storage/storage.go)"]:::storage
        InMem["InMem ChunkStore (pkg/storage/inmemchunkstore)"]:::storage
        Statestore["State Store (pkg/statestore/leveldb/leveldb.go)"]:::storage
    end

    subgraph "External Systems"
        Client["Clients (HTTP/JSON)"]:::external
        Peer["P2P Peers (libp2p)"]:::external
        Ethereum["Ethereum Node (JSON-RPC)"]:::external
    end

    %% Connections
    CLI -->|provides config| Config
    CLI -->|starts| Core
    Config -->|injects| Core

    Core -->|serves| API
    API -->|calls modules| Pull
    API -->|calls modules| Push
    API -->|calls modules| Retrieval
    API -->|calls modules| Postage
    API -->|calls modules| Accounting
    API -->|calls modules| Settlement
    API -->|calls modules| Access

    Core -->|initializes| P2PLC
    Core -->|initializes| Discovery
    Core -->|initializes| Storage
    Core -->|initializes| Statestore
    Core -->|initializes| InMem

    P2PLC -->|discovers via| Discovery
    P2PLC <-->|libp2p streams| Peer
    Pull -->|fetch chunks| P2PLC
    Push -->|send chunks| P2PLC
    Pull -->|store| Storage
    Push -->|store| Storage
    Retrieval -->|read| Storage
    Retrieval -->|read| Statestore

    Listener -->|subscribes to| Ethereum
    Settlement -->|interacts with| Ethereum
    Postage -->|issues stamps| Statestore
    Accounting -->|logs metrics| Metrics

    Logger -->|collects| Core
    Logger -->|collects| API
    Metrics -->|exposes| API
    Metrics -->|monitors| Services

    Client -->|HTTP/JSON| API

    %% Click Events
    click CLI "https://github.com/ethersphere/bee/tree/master/cmd/bee"
    click Config "https://github.com/ethersphere/bee/tree/master/pkg/config"
    click Core "https://github.com/ethersphere/bee/blob/master/pkg/node/node.go"
    click API "https://github.com/ethersphere/bee/tree/master/pkg/api"
    click OpenAPI "https://github.com/ethersphere/bee/blob/master/openapi/Swarm.yaml"
    click P2PLC "https://github.com/ethersphere/bee/tree/master/pkg/p2p/libp2p"
    click Discovery "https://github.com/ethersphere/bee/blob/master/pkg/discovery/discovery.go"
    click Storage "https://github.com/ethersphere/bee/blob/master/pkg/storage/storage.go"
    click InMem "https://github.com/ethersphere/bee/tree/master/pkg/storage/inmemchunkstore"
    click Statestore "https://github.com/ethersphere/bee/blob/master/pkg/statestore/leveldb/leveldb.go"
    click Pull "https://github.com/ethersphere/bee/blob/master/pkg/pullsync/pullsync.go"
    click Push "https://github.com/ethersphere/bee/blob/master/pkg/pushsync/pushsync.go"
    click Retrieval "https://github.com/ethersphere/bee/blob/master/pkg/retrieval/retrieval.go"
    click Postage "https://github.com/ethersphere/bee/blob/master/pkg/postage/service.go"
    click Listener "https://github.com/ethersphere/bee/blob/master/pkg/postage/listener/listener.go"
    click Accounting "https://github.com/ethersphere/bee/blob/master/pkg/accounting/accounting.go"
    click Settlement "https://github.com/ethersphere/bee/tree/master/pkg/settlement/swap"
    click Pseudo "https://github.com/ethersphere/bee/tree/master/pkg/settlement/pseudosettle"
    click Access "https://github.com/ethersphere/bee/blob/master/pkg/accesscontrol/controller.go"
    click Logger "https://github.com/ethersphere/bee/blob/master/pkg/log/logger.go"
    click Metrics "https://github.com/ethersphere/bee/blob/master/pkg/metrics/metrics.go"

    %% Styles
    classDef cli fill:#f9f,stroke:#333,stroke-width:1px
    classDef config fill:#ffd,stroke:#333,stroke-width:1px
    classDef core fill:#ccf,stroke:#333,stroke-width:1px
    classDef api fill:#fcf,stroke:#333,stroke-width:1px
    classDef service fill:#fea,stroke:#333,stroke-width:1px
    classDef support fill:#eee,stroke:#333,stroke-width:1px
    classDef network fill:#bbf,stroke:#333,stroke-width:1px
    classDef storage fill:#bfb,stroke:#333,stroke-width:1px
    classDef external fill:#ddd,stroke:#333,stroke-width:1px
```
