How paritytech/substrate works
Project type and purpose
• This is a monorepo SDK (“Polkadot SDK”) that merges three blockchain‑framework repositories (Cumulus, Substrate, Polkadot).
• It provides everything needed to build a blockchain node: runtime modules (FRAME pallets), client libraries (networking, consensus, storage, executor), node binaries, CLI tools, RPC servers, offchain workers, key management, and developer utilities.
File structure
• Top‑level groupings:
• bin/ → executable templates and production nodes (node‑template, node, chain‑spec tools).
• client/ → client‑side Rust crates: consensus protocols, networking stack, RPC API/backend, service orchestration, transaction pool, executor, keystore, offchain.
• frame/ → FRAME pallets (runtime modules) packaged as individual Cargo crates.
• primitives/ → shared types, runtime‑interface, cryptography, core blockchain primitives.
• utils/, scripts/, docker/ → developer tools, CI/CD pipelines, container definitions, monitoring setup.
• docs/ → contribution guidelines, design docs, style guide.
Key README insight
• The README simply announces the monorepo consolidation; no internal architecture docs.
• Node Binary (e.g. bin/node-template, bin/node) – entry point
• Service Layer (client/service) – orchestrator, task manager, chain‑ops
• Consensus Engines (client/consensus/{aura,babe,grandpa,beefy,…})
• Networking Stack (client/network + client/network‑gossip + client/network‑sync + client/network‑transactions + libp2p transport)
• Transaction Pool (client/transaction-pool)
• Block Builder & Mempool
• Executor (client/executor, client/executor/wasmtime) – runs WASM/native runtime
• Runtime Modules (frame/*) compiled to WASM and linked natively
• State Storage (client/db, client/state-db, primitives/state-machine)
• RPC Layer (client/rpc-api + client/rpc-servers + rpc-spec-v2)
• Offchain Workers & APIs (client/offchain)
• Keystore (client/keystore)
• Telemetry & Metrics (client/telemetry + Prometheus / Grafana containers)
• Developer Tools (subkey, chain-spec-builder, utils/*)
• CLI / RPC requests → Service Layer
• Service → Transaction Pool → Consensus → Block Builder
• Service → Executor (WASM/native) → State Storage
• Networking ↔ Service: broadcast/receive blocks & transactions
• Offchain Workers → Offchain DB / HTTP endpoints
• Service → RPC Layer → External clients
• Service → Telemetry → Prometheus → Grafana
c. Architectural patterns & principles
• Layered architecture (Entry → Service → Components → Storage)
• Plugin‑style modularity (FRAME pallets loaded into runtime)
• Separation of concerns (consensus / network / storage / execution / API)
• Cargo workspaces and crate‑per‑component for strict boundaries
d. Key technologies & libraries
• Rust + Cargo workspaces
• WebAssembly (WASM) for runtime
• libp2p for P2P transport
• Tokio for async
• ParityDB / RocksDB / custom DB backend
• Prometheus + Grafana for metrics
• Docker / Docker Compose for local test networks