How farcasterxyz/hub-monorepo works
Main Components
a. Core Applications:
• Hubble Hub (Primary Application)
• Shuttle (Event Streaming Service)
• Client Libraries (Hub-NodeJS and Hub-Web)
b. External Services:
• Ethereum Network (for on-chain events)
• RocksDB (for storage)
• PostgreSQL (for Shuttle)
• Redis (for Shuttle)
Key Architectural Layers
a. Network Layer:
• P2P Communication (src/network/p2p)
• Sync Engine (src/network/sync)
• RPC Services (src/rpc)
b. Storage Layer:
• RocksDB Integration (src/storage/db)
• Store Management (src/storage/stores)
• Message Processing (src/storage/engine)
c. Protocol Layer:
• Protocol Buffers (protobufs/schemas)
• Message Validation
• Crypto Operations (src/crypto)
[External Clients] <---> [Client Libraries (hub-nodejs/hub-web)]
• v
[Ethereum Network] <---> [Hubble Hub]
• v
[P2P Network] <----------> [Network Layer]
• v
• [Protocol Layer]
• v
• [Storage Layer]
• [RocksDB/Redis]
• v
• [Shuttle Event Streaming]
• v
• [PostgreSQL]
Key Interactions to Highlight
a. Message Flow:
• Client message submission
• P2P message propagation
• Message validation and storage
• Event streaming to Shuttle
b. Synchronization:
• P2P network sync
• Merkle trie synchronization
• Event reconciliation
c. Data Storage:
• Message persistence
• State management
• Event logging
Additional Elements
a. Security Components:
• Ed25519 and EIP712 signers
• Verification systems
• Rate limiting
b. Monitoring:
• Grafana integration
• StatsD metrics
• Health checks
Implementation Details
• The modular architecture with clear separation of concerns
• The hybrid implementation using TypeScript and Rust
• The event-driven nature of the system
• The scalability and reliability features