How samdvr/milena works
• milena-cache – The cache node implementation with multi-tiered storage (in-memory LRU caching, disk store, and backup via S3)
• milena-router – The routing service that uses consistent hashing to distribute client requests across cache nodes and enforce rate limiting
• milena-protos – The collection of Protocol Buffer (.proto) definitions and build scripts for generating gRPC interfaces between the cache and router
Main Components
– Client: The external user or application that sends requests.
– Router: Acts as the gateway to the cache nodes. It’s responsible for using consistent hashing for key distribution among the cache nodes, and it implements rate limiting.
– Cache Nodes: Represented as individual milena-cache instances. Each node implements a multi-tiered storage system:
○ LRU in-memory cache (fast, transient storage)
○ Disk-based persistent storage (for longer-term storage)
○ S3 storage for backup/overflow
– Protocol Definitions: Indicate the gRPC interfaces used to communicate between the Router and Cache Nodes.
Out Relationships and Interactions
– The Client sends requests (Get, Put, Delete) to the Router.
– The Router processes the requests using its consistent hashing mechanism to determine the correct Cache Node.
– Each Cache Node receives requests via gRPC (as defined in the milena-protos) and then handles the request by interacting with its layered storage:
○ It first checks the LRU memory cache.
○ If the data is not found, it looks up the disk store.
○ Optionally, S3 is used as a backup store for additional durability.
– Highlight the feedback loops for monitoring (Prometheus metrics are exposed by cache nodes via a designated endpoint) and error handling.
Architectural Patterns and Technologies
– Distributed System Design: Emphasize the separation of concerns between routing and caching.
– Multi-tiered Caching: Illustrate the hierarchy of caching from in-memory to persistent storage.
Final Touch and Recommendations
– High detail is favored. The more granular you separate the caching tiers within the Cache Node, the better the insights for troubleshooting and scalability planning.