How netflix/evcache works
1. Repository shape
- This is a multi-module Gradle Java project for EVCache, with one shared caching library (evcache-core), a client integration layer (evcache-client), an optional tracing add-on (evcache-zipkin-tracing), a REST proxy service (evcacheproxy), and small sample apps.
2. Primary runtime model
- The core abstraction is a memcached-backed distributed cache.
- Application code talks to EVCache APIs, which route requests to the right cache node via client pools, hashing, and service discovery.
- The underlying transport is based on spymemcached and EVCache-specific extensions under the net.spy.memcached package tree.
3. evcache-core: the main cache engine
- This module contains the public EVCache API (EVCache, EVCacheImpl, EVCacheInternal) plus the bulk of the caching logic.
- It owns cache operations, async futures/latches, key handling, serialization, bulk reads, and exception types.
- It is also where cache behavior is assembled around lower-level memcached clients, node locators, server-group iteration, and retry/fallback utilities.
4. Cluster routing and topology
- The pool layer in core is responsible for choosing which memcached server group/node receives a key.
- Important pieces here are EVCacheClientPool, EVCacheClient, EVCacheNodeLocator, HashRingAlgorithm, ZoneFallbackIterator, and the various server-group iterators.
- This indicates the cache is designed for distributed, zone-aware routing with failover across server groups rather than a single cache endpoint.
5. Service discovery boundary
- Node membership can come from Eureka or a simpler static provider.
- evcache-core includes SimpleNodeListProvider; evcache-client adds EurekaNodeListProvider and related DI wiring.
- Architecture-wise, this is the boundary between topology discovery and request execution: the cluster view changes independently of cache operations.
6. evcache-client: dependency injection and client setup
- This module packages the runtime wiring used by applications that consume EVCache.
- It provides Guice-style modules and connection factories (EVCacheModule, DIConnectionModule, DIConnectionFactory, DIAsciiConnectionFactory) that assemble the core library into a usable client.
- The version package suggests runtime version tracking is exposed as part of client management/observability.
7. Optional tracing integration
- evcache-zipkin-tracing is a separate add-on that listens to EVCache events and emits Zipkin tracing tags/metadata.
- This is not the cache engine itself; it sits on the observability side and augments request visibility without changing cache routing.
8. Eventing, metrics, and management
- Core includes explicit event and listener types (EVCacheEvent, EVCacheEventListener, hot-key and throttle listeners), plus metrics and MBeans.
- There are JMX-facing interfaces for clients, executors, and connection observers, which suggests a strong operational model centered on runtime inspection and tuning.
- Metrics, events, and tracing form the main observability path around the cache.
9. Serialization and value handling
- EVCache supports transcoders and special value wrappers such as EVCacheSerializingTranscoder, EVCacheTranscoder, ChunkTranscoder, and DTO/value classes.
- This shows the cache layer is responsible not only for transport but also for encoding/decoding payloads, including bulk and metadata-rich operations.
10. evcacheproxy: separate deployable service
- The proxy is a distinct web/service application, packaged with Tomcat, JSP/web.xml, properties, and startup scripts.
- It exposes a REST service (EVCacheRESTService) and health check handler, with a StartServer bootstrap and a status page.
- Architecturally, this is a control/access layer over EVCache rather than the core client library.
11. Deployment/runtime concerns in the proxy
- The proxy includes environment-specific property files, a keystore, and Tomcat configuration, indicating it is meant to be deployed as a managed server component.
- The presence of health checks and server startup code implies it participates in service operations, likely for administration or gateway-style access to cache functions.
12. Samples and validation
- evcache-client-sample demonstrates direct client usage, including a Zipkin-tracing sample.
- Test sources in core and client include mock and Eureka-based tests, showing the architecture is exercised both for static cluster behavior and discovery-driven behavior.
13. Docs and build/quality pipeline
- The repository includes MkDocs content (docs/, mkdocs.yml) and GitHub Actions workflows for CI, snapshot, and publish.
- Code quality is enforced through Checkstyle and shared license/header configuration.
- These are supporting systems, but they reinforce that the project is structured as a library plus service artifacts with release automation.
14. High-level data flow to draw
- A client application uses the EVCache client module.
- The client resolves cluster membership through Eureka or static node lists, selects a node via consistent hashing/topology rules, and sends memcached operations through EVCache/spymemcached connections.
- Results, misses, retries, and bulk operations flow back through futures/latches and can emit events, metrics, and optional Zipkin traces.
- The proxy is a separate path that exposes cache operations over HTTP on top of the same EVCache concepts.
Main entry points: EVCacheImpl.java Core Impl · EVCacheClientPool.java Client Pool · EVCacheModule.java Client Module · EVCacheRESTService.java REST Service
Components 19
Core cache
Shared EVCache engine, routing, operations, and memcached integration.
- EVCache API · Primary entry point for cache access and internal implementation.
evcache-core/src/main/java/com/netflix/evcache/EVCache.java - Core Impl · Implements cache operations, latches, misses, and in-memory behavior.
evcache-core/src/main/java/com/netflix/evcache/EVCacheImpl.java - Internal API · Internal extension points and implementation plumbing for EVCache.
evcache-core/src/main/java/com/netflix/evcache/EVCacheInternal.java - Operations · Bulk gets, futures, latches, operation futures, and response objects.
evcache-core/src/main/java/com/netflix/evcache/operation/EVCacheFuture.java - Serialization · Transcoders, DTOs, and value wrappers for payload encoding/decoding.
evcache-core/src/main/java/com/netflix/evcache/EVCacheSerializingTranscoder.java - Client Pool · Coordinates clients, pools, and server-group selection for keys.
evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheClientPool.java - Routing Rules · Node locator, hash ring, fallback iterators, and server-group traversal.
evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheNodeLocator.java - Static Discovery · Simple node list provider used when cluster membership is not Eureka-driven.
evcache-core/src/main/java/com/netflix/evcache/pool/SimpleNodeListProvider.java - Memcached Bindings · EVCache-specific connection, node, and protocol implementations under net.spy.memcached.
evcache-core/src/main/java/net/spy/memcached/EVCacheMemcachedClient.java - Events & Metrics · Events, listeners, metrics factory, and operational MBeans for runtime insight.
evcache-core/src/main/java/com/netflix/evcache/event/EVCacheEvent.java
Client wiring
Application-facing module setup, discovery, and runtime connection factories.
- Client Module · Guice-style module wiring that assembles EVCache for applications.
evcache-client/src/main/java/com/netflix/evcache/EVCacheModule.java - Connection Setup · ASCII and generic DI connection factories plus builder support.
evcache-client/src/main/java/com/netflix/evcache/connection/DIConnectionFactory.java - Eureka Discovery · Eureka-backed node list provider and DI bridge for dynamic cluster membership.
evcache-client/src/main/java/com/netflix/evcache/pool/eureka/EurekaNodeListProvider.java - Version Tracker · Tracks client/runtime version information for management and observability.
evcache-client/src/main/java/com/netflix/evcache/version/VersionTracker.java
Proxy service
Separate deployable REST/Tomcat service built on EVCache concepts.
- Proxy Boot · Tomcat/server startup, health check, and module wiring for the REST proxy.
evcacheproxy/src/main/java/com/netflix/evcache/service/StartServer.java - REST Service · REST endpoint layer and transcoder used to expose EVCache over HTTP.
evcacheproxy/src/main/java/com/netflix/evcache/service/resources/EVCacheRESTService.java - Status Page · Simple status page and health-oriented service presentation layer.
evcacheproxy/src/main/java/com/netflix/evcservice/service/StatusPage.java
Add-ons
Optional tracing and sample usage paths.
- Zipkin Tracing · Event listener that emits Zipkin tags and metadata for EVCache activity.
evcache-zipkin-tracing/src/main/java/com/netflix/evcache/EVCacheTracingEventListener.java - Samples · Direct client and tracing sample applications showing usage patterns.
evcache-client-sample/src/main/java/com/netflix/evcache/sample/EVCacheClientSample.java
Connections 21
- EVCache API → Core Impl: implements · Public API is realized by the core implementation.
- Core Impl → Operations: uses · Cache operations delegate to async futures, latches, and bulk read handling.
- Core Impl → Serialization: encodes values · Payloads are wrapped and transcoded before transport.
- Core Impl → Client Pool: routes via · Implementation relies on the client pool to choose the correct cache target.
- Internal API → Client Pool: orchestrates · Internal API coordinates lower-level pool behavior and execution boundaries.
- Client Pool → Routing Rules: selects node · Pool consults hashing and node locator rules to place keys.
- Client Pool → Static Discovery: reads topology · Static topology provider supplies server membership when not using discovery.
- Routing Rules → Memcached Bindings: targets · Routing rules resolve to EVCache-specific memcached nodes and connections.
- Memcached Bindings → Operations: executes ops · Memcached client/protocol layer carries cache operations to servers.
- Events & Metrics → Core Impl: observes · Event and metrics layer wraps cache activity and runtime state.
- Client Module → Connection Setup: assembles · Client module wires connection factories into application runtime.
- Client Module → Eureka Discovery: injects · Client module binds Eureka-based topology discovery.
- Eureka Discovery → Client Pool: supplies topology · Discovery feeds the pool with current server-group membership.
- Version Tracker → Client Module: reports · Version metadata is exposed alongside client runtime management.
- Zipkin Tracing → Events & Metrics: listens to · Tracing integrates through EVCache events rather than core routing.
- Samples → Client Module: demonstrates · Sample apps show direct client usage and tracing integration.
- Proxy Boot → REST Service: starts · Bootstrap launches the REST service inside the proxy runtime.
- Proxy Boot → Status Page: hosts · Service bootstrap also serves health and status endpoints.
- REST Service → EVCache API: calls · REST layer uses the same EVCache concepts and operations underneath.
- REST Service → Serialization: transcodes · Proxy service uses its own transcoder for HTTP payload handling.
- Status Page → Proxy Boot: monitors · Status view reflects the proxy server lifecycle.