# netflix/evcache architecture

> Architecture diagram of the GitHub repository netflix/evcache, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/netflix/evcache
- Repository: https://github.com/netflix/evcache

## Overview

1. **Repository shape**
   - This is a multi-module **Gradle** Java project for EVCache, with one shared caching library (`evcache-core`), a client integration layer (`evcache-client`), an optional tracing add-on (`evcache-zipkin-tracing`), a REST proxy service (`evcacheproxy`), and small sample apps.

2. **Primary runtime model**
   - The core abstraction is a **memcached-backed distributed cache**.
   - Application code talks to EVCache APIs, which route requests to the right cache node via **client pools, hashing, and service discovery**.
   - The underlying transport is based on **spymemcached** and EVCache-specific extensions under the `net.spy.memcached` package tree.

3. **`evcache-core`: the main cache engine**
   - This module contains the public EVCache API (`EVCache`, `EVCacheImpl`, `EVCacheInternal`) plus the bulk of the caching logic.
   - It owns cache operations, async futures/latches, key handling, serialization, bulk reads, and exception types.
   - It is also where cache behavior is assembled around lower-level memcached clients, node locators, server-group iteration, and retry/fallback utilities.

4. **Cluster routing and topology**
   - The pool layer in core is responsible for **choosing which memcached server group/node receives a key**.
   - Important pieces here are `EVCacheClientPool`, `EVCacheClient`, `EVCacheNodeLocator`, `HashRingAlgorithm`, `ZoneFallbackIterator`, and the various server-group iterators.
   - This indicates the cache is designed for **distributed, zone-aware routing** with failover across server groups rather than a single cache endpoint.

5. **Service discovery boundary**
   - Node membership can come from **Eureka** or a simpler static provider.
   - `evcache-core` includes `SimpleNodeListProvider`; `evcache-client` adds `EurekaNodeListProvider` and related DI wiring.
   - Architecture-wise, this is the boundary between **topology discovery** and **request execution**: the cluster view changes independently of cache operations.

6. **`evcache-client`: dependency injection and client setup**
   - This module packages the runtime wiring used by applications that consume EVCache.
   - It provides Guice-style modules and connection factories (`EVCacheModule`, `DIConnectionModule`, `DIConnectionFactory`, `DIAsciiConnectionFactory`) that assemble the core library into a usable client.
   - The `version` package suggests runtime version tracking is exposed as part of client management/observability.

7. **Optional tracing integration**
   - `evcache-zipkin-tracing` is a separate add-on that listens to EVCache events and emits **Zipkin tracing tags/metadata**.
   - This is not the cache engine itself; it sits on the **observability side** and augments request visibility without changing cache routing.

8. **Eventing, metrics, and management**
   - Core includes explicit event and listener types (`EVCacheEvent`, `EVCacheEventListener`, hot-key and throttle listeners), plus metrics and MBeans.
   - There are JMX-facing interfaces for clients, executors, and connection observers, which suggests a strong operational model centered on **runtime inspection and tuning**.
   - Metrics, events, and tracing form the main observability path around the cache.

9. **Serialization and value handling**
   - EVCache supports transcoders and special value wrappers such as `EVCacheSerializingTranscoder`, `EVCacheTranscoder`, `ChunkTranscoder`, and DTO/value classes.
   - This shows the cache layer is responsible not only for transport but also for **encoding/decoding payloads**, including bulk and metadata-rich operations.

10. **`evcacheproxy`: separate deployable service**
   - The proxy is a distinct web/service application, packaged with **Tomcat**, JSP/web.xml, properties, and startup scripts.
   - It exposes a REST service (`EVCacheRESTService`) and health check handler, with a `StartServer` bootstrap and a status page.
   - Architecturally, this is a **control/access layer** over EVCache rather than the core client library.

11. **Deployment/runtime concerns in the proxy**
   - The proxy includes environment-specific property files, a keystore, and Tomcat configuration, indicating it is meant to be deployed as a managed server component.
   - The presence of health checks and server startup code implies it participates in service operations, likely for administration or gateway-style access to cache functions.

12. **Samples and validation**
   - `evcache-client-sample` demonstrates direct client usage, including a Zipkin-tracing sample.
   - Test sources in core and client include mock and Eureka-based tests, showing the architecture is exercised both for **static cluster behavior** and **discovery-driven behavior**.

13. **Docs and build/quality pipeline**
   - The repository includes MkDocs content (`docs/`, `mkdocs.yml`) and GitHub Actions workflows for CI, snapshot, and publish.
   - Code quality is enforced through Checkstyle and shared license/header configuration.
   - These are supporting systems, but they reinforce that the project is structured as a **library plus service artifacts** with release automation.

14. **High-level data flow to draw**
   - A client application uses the EVCache client module.
   - The client resolves cluster membership through Eureka or static node lists, selects a node via consistent hashing/topology rules, and sends memcached operations through EVCache/spymemcached connections.
   - Results, misses, retries, and bulk operations flow back through futures/latches and can emit events, metrics, and optional Zipkin traces.
   - The proxy is a separate path that exposes cache operations over HTTP on top of the same EVCache concepts.

Main entry points: [`evcache-core/src/main/java/com/netflix/evcache/EVCacheImpl.java`](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheImpl.java) (Core Impl), [`evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheClientPool.java`](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheClientPool.java) (Client Pool), [`evcache-client/src/main/java/com/netflix/evcache/EVCacheModule.java`](https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/EVCacheModule.java) (Client Module), [`evcacheproxy/src/main/java/com/netflix/evcache/service/resources/EVCacheRESTService.java`](https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcache/service/resources/EVCacheRESTService.java) (REST Service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_core["Core cache"]
  node_node_evcache_api["EVCache API<br/>public cache api<br/>[EVCache.java]"]
  node_node_evcache_impl["Core Impl<br/>cache engine<br/>[EVCacheImpl.java]"]
  node_node_evcache_internal["Internal API<br/>internal control"]
  node_node_operations["Operations<br/>async futures<br/>[EVCacheFuture.java]"]
  node_node_serialization["Serialization<br/>value encoding"]
  node_node_pool["Client Pool<br/>cluster routing"]
  node_node_routing["Routing Rules<br/>hashing/topology"]
  node_node_discovery_core["Static Discovery<br/>node list provider"]
  node_node_memcached["Memcached Bindings<br/>spymemcached extensions"]
  node_node_metrics_events["Events & Metrics<br/>observability<br/>[EVCacheEvent.java]"]
end

subgraph group_group_client["Client wiring"]
  node_node_client_module["Client Module<br/>di wiring<br/>[EVCacheModule.java]"]
  node_node_connection["Connection Setup<br/>connection factories"]
  node_node_discovery_client["Eureka Discovery<br/>service discovery"]
  node_node_version["Version Tracker<br/>runtime metadata"]
end

subgraph group_group_proxy["Proxy service"]
  node_node_proxy_boot["Proxy Boot<br/>web service bootstrap<br/>[StartServer.java]"]
  node_node_proxy_rest["REST Service<br/>http proxy"]
  node_node_proxy_status["Status Page<br/>admin ui<br/>[StatusPage.java]"]
end

subgraph group_group_addons["Add-ons"]
  node_node_tracing["Zipkin Tracing<br/>optional tracing"]
  node_node_samples["Samples<br/>example apps"]
end

node_node_evcache_api -->|"implements"| node_node_evcache_impl
node_node_evcache_impl -->|"uses"| node_node_operations
node_node_evcache_impl -->|"encodes values"| node_node_serialization
node_node_evcache_impl -->|"routes via"| node_node_pool
node_node_evcache_internal -->|"orchestrates"| node_node_pool
node_node_pool -->|"selects node"| node_node_routing
node_node_pool -->|"reads topology"| node_node_discovery_core
node_node_routing -->|"targets"| node_node_memcached
node_node_memcached -->|"executes ops"| node_node_operations
node_node_metrics_events -->|"observes"| node_node_evcache_impl
node_node_client_module -->|"assembles"| node_node_connection
node_node_client_module -->|"injects"| node_node_discovery_client
node_node_discovery_client -->|"supplies topology"| node_node_pool
node_node_version -->|"reports"| node_node_client_module
node_node_tracing -->|"listens to"| node_node_metrics_events
node_node_samples -->|"demonstrates"| node_node_client_module
node_node_proxy_boot -->|"starts"| node_node_proxy_rest
node_node_proxy_boot -->|"hosts"| node_node_proxy_status
node_node_proxy_rest -->|"calls"| node_node_evcache_api
node_node_proxy_rest -->|"transcodes"| node_node_serialization
node_node_proxy_status -->|"monitors"| node_node_proxy_boot

click node_node_evcache_api "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCache.java"
click node_node_evcache_impl "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheImpl.java"
click node_node_evcache_internal "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheInternal.java"
click node_node_operations "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/operation/EVCacheFuture.java"
click node_node_serialization "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheSerializingTranscoder.java"
click node_node_pool "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheClientPool.java"
click node_node_routing "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheNodeLocator.java"
click node_node_discovery_core "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/SimpleNodeListProvider.java"
click node_node_memcached "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/net/spy/memcached/EVCacheMemcachedClient.java"
click node_node_metrics_events "https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/event/EVCacheEvent.java"
click node_node_client_module "https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/EVCacheModule.java"
click node_node_connection "https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/connection/DIConnectionFactory.java"
click node_node_discovery_client "https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/pool/eureka/EurekaNodeListProvider.java"
click node_node_version "https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/version/VersionTracker.java"
click node_node_tracing "https://github.com/netflix/evcache/blob/master/evcache-zipkin-tracing/src/main/java/com/netflix/evcache/EVCacheTracingEventListener.java"
click node_node_samples "https://github.com/netflix/evcache/blob/master/evcache-client-sample/src/main/java/com/netflix/evcache/sample/EVCacheClientSample.java"
click node_node_proxy_boot "https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcache/service/StartServer.java"
click node_node_proxy_rest "https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcache/service/resources/EVCacheRESTService.java"
click node_node_proxy_status "https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcservice/service/StatusPage.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_evcache_api,node_node_evcache_impl,node_node_evcache_internal,node_node_operations,node_node_serialization,node_node_pool,node_node_routing,node_node_discovery_core,node_node_memcached,node_node_metrics_events toneBlue
class node_node_client_module,node_node_connection,node_node_discovery_client,node_node_version toneAmber
class node_node_proxy_boot,node_node_proxy_rest,node_node_proxy_status toneMint
class node_node_tracing,node_node_samples toneRose
```

## Components

### Core cache

Shared EVCache engine, routing, operations, and memcached integration.

- [EVCache API](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCache.java): public cache api · Primary entry point for cache access and internal implementation. · `evcache-core/src/main/java/com/netflix/evcache/EVCache.java`
- [Core Impl](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheImpl.java): cache engine · Implements cache operations, latches, misses, and in-memory behavior. · `evcache-core/src/main/java/com/netflix/evcache/EVCacheImpl.java`
- [Internal API](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheInternal.java): internal control · Internal extension points and implementation plumbing for EVCache. · `evcache-core/src/main/java/com/netflix/evcache/EVCacheInternal.java`
- [Operations](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/operation/EVCacheFuture.java): async futures · Bulk gets, futures, latches, operation futures, and response objects. · `evcache-core/src/main/java/com/netflix/evcache/operation/EVCacheFuture.java`
- [Serialization](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/EVCacheSerializingTranscoder.java): value encoding · Transcoders, DTOs, and value wrappers for payload encoding/decoding. · `evcache-core/src/main/java/com/netflix/evcache/EVCacheSerializingTranscoder.java`
- [Client Pool](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheClientPool.java): cluster routing · Coordinates clients, pools, and server-group selection for keys. · `evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheClientPool.java`
- [Routing Rules](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheNodeLocator.java): hashing/topology · Node locator, hash ring, fallback iterators, and server-group traversal. · `evcache-core/src/main/java/com/netflix/evcache/pool/EVCacheNodeLocator.java`
- [Static Discovery](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/pool/SimpleNodeListProvider.java): node list provider · Simple node list provider used when cluster membership is not Eureka-driven. · `evcache-core/src/main/java/com/netflix/evcache/pool/SimpleNodeListProvider.java`
- [Memcached Bindings](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/net/spy/memcached/EVCacheMemcachedClient.java): spymemcached extensions · EVCache-specific connection, node, and protocol implementations under net.spy.memcached. · `evcache-core/src/main/java/net/spy/memcached/EVCacheMemcachedClient.java`
- [Events & Metrics](https://github.com/netflix/evcache/blob/master/evcache-core/src/main/java/com/netflix/evcache/event/EVCacheEvent.java): observability · Events, listeners, metrics factory, and operational MBeans for runtime insight. · `evcache-core/src/main/java/com/netflix/evcache/event/EVCacheEvent.java`

### Client wiring

Application-facing module setup, discovery, and runtime connection factories.

- [Client Module](https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/EVCacheModule.java): di wiring · Guice-style module wiring that assembles EVCache for applications. · `evcache-client/src/main/java/com/netflix/evcache/EVCacheModule.java`
- [Connection Setup](https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/connection/DIConnectionFactory.java): connection factories · ASCII and generic DI connection factories plus builder support. · `evcache-client/src/main/java/com/netflix/evcache/connection/DIConnectionFactory.java`
- [Eureka Discovery](https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/pool/eureka/EurekaNodeListProvider.java): service discovery · Eureka-backed node list provider and DI bridge for dynamic cluster membership. · `evcache-client/src/main/java/com/netflix/evcache/pool/eureka/EurekaNodeListProvider.java`
- [Version Tracker](https://github.com/netflix/evcache/blob/master/evcache-client/src/main/java/com/netflix/evcache/version/VersionTracker.java): runtime metadata · Tracks client/runtime version information for management and observability. · `evcache-client/src/main/java/com/netflix/evcache/version/VersionTracker.java`

### Proxy service

Separate deployable REST/Tomcat service built on EVCache concepts.

- [Proxy Boot](https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcache/service/StartServer.java): web service bootstrap · Tomcat/server startup, health check, and module wiring for the REST proxy. · `evcacheproxy/src/main/java/com/netflix/evcache/service/StartServer.java`
- [REST Service](https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcache/service/resources/EVCacheRESTService.java): http proxy · REST endpoint layer and transcoder used to expose EVCache over HTTP. · `evcacheproxy/src/main/java/com/netflix/evcache/service/resources/EVCacheRESTService.java`
- [Status Page](https://github.com/netflix/evcache/blob/master/evcacheproxy/src/main/java/com/netflix/evcservice/service/StatusPage.java): admin ui · Simple status page and health-oriented service presentation layer. · `evcacheproxy/src/main/java/com/netflix/evcservice/service/StatusPage.java`

### Add-ons

Optional tracing and sample usage paths.

- [Zipkin Tracing](https://github.com/netflix/evcache/blob/master/evcache-zipkin-tracing/src/main/java/com/netflix/evcache/EVCacheTracingEventListener.java): optional tracing · Event listener that emits Zipkin tags and metadata for EVCache activity. · `evcache-zipkin-tracing/src/main/java/com/netflix/evcache/EVCacheTracingEventListener.java`
- [Samples](https://github.com/netflix/evcache/blob/master/evcache-client-sample/src/main/java/com/netflix/evcache/sample/EVCacheClientSample.java): example apps · Direct client and tracing sample applications showing usage patterns. · `evcache-client-sample/src/main/java/com/netflix/evcache/sample/EVCacheClientSample.java`

## Connections

- EVCache API → Core Impl: implements · Public API is realized by the core implementation.
- Core Impl → Operations: uses · Cache operations delegate to async futures, latches, and bulk read handling.
- Core Impl → Serialization: encodes values · Payloads are wrapped and transcoded before transport.
- Core Impl → Client Pool: routes via · Implementation relies on the client pool to choose the correct cache target.
- Internal API → Client Pool: orchestrates · Internal API coordinates lower-level pool behavior and execution boundaries.
- Client Pool → Routing Rules: selects node · Pool consults hashing and node locator rules to place keys.
- Client Pool → Static Discovery: reads topology · Static topology provider supplies server membership when not using discovery.
- Routing Rules → Memcached Bindings: targets · Routing rules resolve to EVCache-specific memcached nodes and connections.
- Memcached Bindings → Operations: executes ops · Memcached client/protocol layer carries cache operations to servers.
- Events & Metrics → Core Impl: observes · Event and metrics layer wraps cache activity and runtime state.
- Client Module → Connection Setup: assembles · Client module wires connection factories into application runtime.
- Client Module → Eureka Discovery: injects · Client module binds Eureka-based topology discovery.
- Eureka Discovery → Client Pool: supplies topology · Discovery feeds the pool with current server-group membership.
- Version Tracker → Client Module: reports · Version metadata is exposed alongside client runtime management.
- Zipkin Tracing → Events & Metrics: listens to · Tracing integrates through EVCache events rather than core routing.
- Samples → Client Module: demonstrates · Sample apps show direct client usage and tracing integration.
- Proxy Boot → REST Service: starts · Bootstrap launches the REST service inside the proxy runtime.
- Proxy Boot → Status Page: hosts · Service bootstrap also serves health and status endpoints.
- REST Service → EVCache API: calls · REST layer uses the same EVCache concepts and operations underneath.
- REST Service → Serialization: transcodes · Proxy service uses its own transcoder for HTTP payload handling.
- Status Page → Proxy Boot: monitors · Status view reflects the proxy server lifecycle.