How eqlabs/pathfinder works
Project type and purpose
• Pathfinder is a Rust‐based full node implementation for the Starknet blockchain. It ingests and gossips blocks via a P2P network, runs a consensus protocol (“Malachite”), executes transactions to update state, persists everything to a database, and exposes a JSON-RPC/WebSocket API to clients.
Key architectural layers (from bottom to top)
a. Storage Layer
• Crate “storage”: handles database connections (SQLite or Postgres), schemas, migrations, and low-level CRUD for blocks, states, receipts, transactions, events, proofs, etc.
• Optional “fake” adapters and test fixtures under “crates/storage/src/fake.rs” and “crates/storage/src/test_utils.rs.”
b. Execution Layer
• Crate “executor”: applies transaction classes to state, tracks pending calls, caches contract classes, and simulates calls.
• Crate “block-hashes” and “class-hash”: helper crates for computing deterministic hash functions required by the Starknet VM.
c. Consensus Layer
• Crate “consensus”: implements the Malachite consensus protocol modules (proposals, rounds, votes, WAL).
• Crate “p2p” + “p2p_proto” + “p2p_stream”: networking transport, message codecs (protobuf), and high-level P2P behaviors for consensus gossip and sync.
d. Gateway / L1 Integration
• Crate “gateway-client”: pulls new L1 state updates and blocks from a Starknet gateway (HTTP/JSON), used during initial sync or state updates.
• Crate “gateway-types” and “gateway-test-fixtures” for modeling and testing gateway responses.
e. RPC / API Layer
• Crate “rpc”: exposes JSON-RPC and WebSocket endpoints, routes methods (get_block, get_state_update, simulate, subscribe_new_heads, etc.), embeds middleware (CORS, tracing), and dispatches to executor, storage, or pathfinder core.
• Specs under “specs/rpc” track OpenRPC definitions per version.
f. Orchestration / Binary
• Crate “pathfinder”: the main binary. It wires together all components, reads configuration (CLI + env), initializes the P2P network, starts consensus, triggers execution and storage subsystems, and launches the RPC server.
• “build/” & Docker assets for containerized builds and deployments.
External services and integrations
• Ethereum L1 node or Starknet sequencer (via gateway-client) for pulling blocks and L1 events
• Clients connecting via JSON-RPC / WebSockets
• Prometheus / metrics (see “metrics.rs” in gateway-client and “monitoring.rs” in pathfinder)
• Configuration / CLI inputs
• Main Orchestrator (pathfinder binary)
• P2P ↔ Consensus: block gossip, votes, sync messages
• Consensus → Orchestrator: new finalized block events
• Orchestrator → Executor: execute block transactions
• Executor → Storage: persist new state, receipts, block data
• Gateway Client → Orchestrator: push L1 updates for state sync
• RPC Server ↔ Orchestrator/Executor/Storage: client queries mapped to appropriate subsystem
• Monitoring → Metrics Endpoint: scrape metrics
• Blue: Networking components (P2P, Gateway Client)
• Green: Core computation (Consensus, Executor)
• Orange: Data persistence (Storage)
• Purple: External API (RPC Server)
• Grey: Orchestrator/CLI/Config
Architectural patterns and principles
• Modular crate-per-domain approach: each core domain (storage, execution, consensus, RPC) is isolated into its own crate.
• Layered architecture: clear separation of transport (P2P, HTTP) → business logic (consensus, execution) → persistence (storage).
• Pluggable transports and protocol versions: P2P stack uses protocol buffers and can evolve independent of business logic.