How celestiaorg/celestia-core works
This project – celestia-core – is a modified fork of CometBFT (Tendermint core) that serves as the consensus and networking engine for the Celestia ecosystem. It implements the Tendermint protocol with Celestia-specific modifications (for example, adjustments to block data hashing for data availability sampling and the early adoption of ABCI++ methods like PrepareProposal and ProcessProposal). The project supports components you’d expect in a blockchain node, including consensus, networking (p2p), mempool management, state management, cryptographic operations, and integration with the application (via the ABCI interface).
Main Components
• Consensus Engine: Representing the core of the Tendermint consensus mechanism. This includes modules for consensus logic, block validation, vote aggregation, and WAL (write-ahead logging). Look at directories like “consensus” and “blockchain.”
• Networking (P2P): Includes all p2p functionalities such as connection handling, peer management, address book (pex), transport protocols, and node discovery. Directories “p2p” and “networks” will be helpful here.
• Mempool: Manages pending transactions before they are included in blocks. Files in “mempool” detail how transactions are validated, cached, and broadcasted.
• ABCI / State Machine Integration: Although celestia-core runs the consensus engine, it interacts with an external application (celestia-app) via the Application BlockChain Interface (ABCI). The “abci” folder includes client and server implementations. This shows how blocks, proposals, and state transitions are communicated.
• Data Availability Enhancements: Celestia-core modifies the DataHash calculation to commit not just transaction data but also the erasure-coded data square. This is part of the blockchain and consensus layer interaction.
• Cryptography and Utilities: Encompassing the crypto modules (ed25519, secp256k1, merkle trees, etc.) that are essential for signing, hashing, and verifying data. Directories “crypto” and “libs/crypto” play a role.
• API & RPC Interfaces: Providing access via JSON-RPC, gRPC, and other endpoints. Check the “rpc” and “pkg/trace” directories for implementation details.
• Additional Infrastructure: There are deployment, containerization (Dockerfiles), testing (extensive test suites), and build scripts that support development and release (Makefile, Go module files, etc.).
Interactions and Relationships
• That the network layer (P2P) interacts with both the consensus engine (for receiving votes and gossiping block data) and the mempool (to distribute pending transactions).
• That the cryptography modules underpin security in all components (consensus, p2p, mempool).
• Modifications for data availability: the consensus engine now incorporates the erasure-coded data square’s row/column commitments when computing the block’s header (i.e. customized DataHash logic).
Architectural Patterns and Design Principles
• Modular design: Each core functionality (consensus, networking, mempool, state) is separated into its own package or module.
• Client–Server Pattern: The ABCI modules use a clear client–server interaction separating the application state (celestia-app) from the consensus engine.
• Event-driven architecture: The pubsub and event packages (located under libs/pubsub and types/event_bus.go) suggest asynchronous event processing.
• Layered architecture: Lower-level libraries (e.g., cryptography, networking) support higher-level consensus and state-management components.
Relevant Technologies/Frameworks
• Go as the primary language with common Go tooling (Makefile, go.mod, etc.)
• Docker for containerization (see the DOCKER directory and docker-compose.yml).
• gRPC, JSON-RPC for API connectivity, as found in the “rpc” directory.
• Protobuf (with buf and proto directories) is used to generate protocol files for consensus and networking communication.
• Use clear labels for each component (e.g., “Consensus Engine,” “P2P Networking,” “Mempool,” “ABCI Bridge / Application Interface,” “Cryptographic Modules,” “RPC/API Layer”).
• Feedback loops in the consensus process (e.g., block commit, WAL logging).
• Using grouping or container symbols to explicitly show which modules are part of the “celestia-core” system versus external systems like “celestia-app.”
Happy diagramming!