How ipfs/boxo works
Project type and purpose
• This is a monorepo for Boxo, an IPFS SDK in Go. It provides a suite of libraries (Bitswap, BlockService, Blockstore, IPLD, UnixFS, Routing, Gateway, IPNS, Pinning, etc.) plus CLI tools and examples to build IPFS-aware applications or reimplement IPFS features.
File structure
• Top-level packages represent major functional domains:
– network & exchange: bitswap, exchange, provider, routing
– storage & retrieval: blockservice, blockstore, datastore, filestore, fetcher
– content addressing & data models: ipld/merkledag, ipld/unixfs, chunker, verifcid
– naming & mutability: ipns, namesys, path
– utilities & supporting: keystore, mfs, files, tar, tracing, util
– façade & integration: gateway, cmd (boxo-migrate, deprecator), examples
• There are also internal packages for private helpers, tests, and build/deployment pipelines (.github/workflows).
README for additional insights
• The README emphasizes Boxo as a “component library” for IPFS in Go, powering Kubo.
• It lists major protocol components (DHT, Bitswap, Gateway, IPNS, etc.) that should be represented.
1. Application Layer (your custom app or CLI commands)
2. Boxo SDK façade
3. Core modules:
• Content Routing (namesys, routing/http, routing/offline)
• Data Transfer (bitswap + network interface)
• BlockService & Blockstore (blockservice ↔ blockstore ↔ datastore)
• IPLD + UnixFS (ipld/merkledag, ipld/unixfs, chunker)
• Naming (ipns)
• Pinning (pinning/pinner, pinning/remote)
• Provider & Reprovider (provider queue)
4. Gateway HTTP Layer (gateway, assets, handler)
5. CLI Tools (cmd/boxo-migrate, cmd/deprecator)
6. Examples (examples/*)
7. External Dependencies:
• IPFS network peers (DHT, Bitswap, HTTP Gateway)
• Underlying datastore (LevelDB, Badger, etc.)
• Clients or other Go modules
b. Relationships & interactions:
– App Layer → Boxo façade API calls
– Façade → Core modules by import/injection
– Content Retrieval: Façade → bitswap → network interface → remote peers → bitswap → blockservice → ipld → unixfs → application
– Content Storage: App → blockservice → blockstore → datastore
– Routing: Façade → routing/http or DHT module → peer discovery
– Naming: App → ipns → namesys → DHT
– Pinning: App → pinner → blockstore
– Gateway: HTTP requests → gateway handlers → blockservice + ipld/unixfs → HTTP responses
– CLI: uses Boxo libraries same as App
c. Architectural patterns/ principles:
– Modular plugin-like packages, each focused on one IPFS protocol or service
– Clear separation of concerns (transport vs. data vs. naming)
– Interfaces for offline vs. online implementations (exchange/interface, routing/interface)
– Dependency injection: network, datastore, tracing exporters
d. Key technologies/frameworks:
– Go modules + pkg structure
– Protobuf for message definitions (bitswap, ipld, ipns)
– HTTP for gateway and delegated routing
– gRPC-style interfaces (routing HTTP client)
– Testing harnesses & benchmarks
Detailed capture of essential elements
Layer 1: App / CLI / Examples
Layer 2: Boxo façade & CLI commands
Layer 3: Protocol modules (routing, bitswap, ipns, provider, pinning)
Layer 4: Data layers (blockservice, blockstore, datastore, ipld/unixfs)
Layer 5: External systems (IPFS peers, DHT, HTTP gateway)