phuslu/lru

1. What this repo is

This is a Go library for in-memory caching, centered on two main cache flavors: a classic sharded LRU cache and a TTL cache variant. It is optimized for hot-path use in services where predictable latency, low GC pressure, and immediate consistency of cache operations matter.

2. Public API surface

The exported entry points are the generic LRUCache[K,V], TTLCache[K,V], and BytesCache implementations. The generic caches support normal Get/Set-style access; the TTL cache adds per-entry expiry; the bytes cache is specialized for []byte keys and values and avoids string conversion and extra copying.

3. Primary runtime boundary: sharding

The core architectural boundary is sharding. Cache capacity is split into fixed-size shards so lookups and updates only touch one shard at a time. This is the main concurrency and scalability mechanism, and it also helps keep per-shard data structures compact and predictable.

4. Per-shard storage model

Each shard is built from two main pieces: a lookup table and a recency list. The table gives fast key-to-entry lookup, while the list maintains LRU order for eviction and promotion on access. The file split into *_shard.go, *_shard_list.go, and *_shard_table.go reflects that separation.

5. LRU data flow

For the non-TTL cache, a Get does two things: it looks up the entry in the shard table and then updates the item’s recency position in the shard list. A Set inserts or replaces the entry and may evict the least-recently-used entry when the shard is full. The behavior is synchronous; there is no background admission or delayed write path.

6. TTL data flow

The TTL cache layers expiration metadata on top of the same sharded cache shape. On access, the shard checks expiry before returning a value. Expired entries are removed lazily, either when accessed again or when capacity pressure forces eviction. The README notes TTL precision is to the nearest second, which is an intentional implementation constraint.

7. Loader and singleflight path

The presence of singleflight.go and the loader option in options.go indicates an optional cache-miss loading path. When enabled, concurrent requests for the same missing key are coalesced so only one load function executes, reducing duplicate backend work. This is a boundary between the cache and an upstream data source, not a separate async subsystem.

8. Bytes-oriented fast path

bytes_cache.go, bytes_shard.go, and related files define a separate implementation optimized for []byte-heavy workloads. Its architectural goal is to keep the byte path zero-copy and avoid allocation-heavy conversions, under the assumption that stored slices are treated as immutable by the caller.

9. Hashing and identity

wyhash.go provides the hashing primitive used to map keys to shards and table slots. This is part of the performance story: fast key hashing reduces overhead on every access and supports the compact shard/table design. For generic keys, the cache relies on deterministic key hashing; for byte keys, it can stay in the byte domain.

10. Clock and expiration support

ttl_clock.go isolates time source behavior for TTL handling. This makes expiry logic a distinct subsystem rather than hard-coding time access throughout the cache. It also supports the library’s stated focus on predictable TTL behavior and testability.

11. Stats and observability

stats.go exposes counters such as set/get/miss activity, which lets users observe cache behavior without instrumenting the internals. This is lightweight observability built into the cache, not an external metrics pipeline.

12. Options and feature toggles

options.go is the configuration boundary for optional behavior such as sliding expiration and loader integration. The repository treats these as feature switches on top of the same core cache engine, rather than separate products. That keeps the architecture centered on one core LRU/TTL implementation with opt-in behaviors.

13. GC and memory design constraints

A major design goal is reducing GC scan cost and heap churn. The README and file layout point to array-backed lists, compact tables, low pointer density, and no per-entry allocation. The “nottl” benchmark target in the README shows the TTL feature is optional and that removing it saves memory, reinforcing that memory layout is an architectural concern, not just an implementation detail.

14. Verification and regression workflow

The GitHub Actions workflows are split by concern: unit tests, benchmarks, GC scan measurement, and memory profiling. That tells you the repo is treated as a performance-sensitive library where correctness, throughput, GC cost, and memory footprint are all first-class release criteria.

15. Diagram takeaway

If you draw the architecture, the key shape is: public cache APIs at the top; optional loader/TTL/statistics around them; a sharded core in the middle; and per-shard table/list storage plus hashing/clock helpers underneath. The main cross-cutting concern is performance under Go’s GC, which drives the choice of fixed-size shards and compact in-memory structures.