How justpranavrs/tlru works
1. What this repository is
- tlru is a Go cache library, not an application.
- The architecture is split into a public facade in the module root and a lower-level core engine. The root package exposes the user-facing cache types; core contains the actual storage and eviction algorithms.
2. Public API layer
- The root files (lru.go, slru.go, tlru.go, pool.go, tpool.go) provide the constructors and option-based configuration for the exported cache types.
- This layer is the main boundary for consumers: they choose between single-instance caches and pooled/sharded caches, and between plain, TTL, and segmented variants.
3. Core cache engine
- core/ holds the fundamental implementations: core.LRU, core.TLRU, core.SLRU, and core.TSLRU.
- These are array-based, generic caches built for fixed capacity and zero runtime allocations. The README indicates they use int32 indices rather than heap-heavy linked nodes, which is the main performance design choice.
4. Single-instance vs sharded architecture
- The library has two runtime shapes:
- Single cache instances in core/, which preserve global eviction behavior.
- Pooled/sharded caches in the root package, which route each key to one shard and scale under concurrency by avoiding a global lock.
- Sharded caches trade perfect global LRU ordering for throughput and reduced contention.
5. Time-to-live subsystem
- TTL behavior lives in core.TLRU / core.TSLRU and the root tlru/tpool wrappers.
- TTL is lazy: expiration is checked on Get and Peek, not by a background cleanup goroutine.
- The model supports both absolute TTL and sliding TTL. A clock abstraction in clock/ lets the cache use a custom time source.
6. Segmented cache subsystem
- core.SLRU and core.TSLRU implement segmented LRU using two internal regions: probationary and protected.
- The promotion path reduces scan pollution and is intended to improve hit rate compared with plain LRU.
- The root slru.go / tpool.go files expose the sharded versions of these caches.
7. Sharding and key routing
- mux/ is the sharding router. It maps a key to a shard index using selectable hash strategies.
- The shipped hashers are fnv, xxh, and maphash, which allows the cache to support different key types and distribution characteristics.
- This router is the key boundary between the public pooled cache and the underlying core instances.
8. Synchronization model
- Concurrency is handled primarily at the shard/cache level rather than by a single global lock.
- core/ includes shared synchronization support in sync.go, while each core cache instance manages its own mutex-protected state.
- This makes the sharded caches naturally scalable while keeping the single-cache implementation simple.
9. Batch and hot-path optimizations
- core.LRU includes batch-oriented operations such as GetMany and PutMany, which are optimized to reduce lock contention.
- The pooled wrappers do not expose the same batch semantics as prominently; the performance focus is on fast single-key operations through shard routing.
10. Internal/test-only utilities
- internal/testutil/ is isolated test scaffolding: deterministic data generation, random data, and shared test setup.
- It does not participate in runtime architecture; it exists to support correctness tests and benchmarks across the cache variants.
11. Repository structure and validation
- The *_test.go and *_example_test.go files are used to verify behavior and document usage patterns.
- CI is defined in .github/workflows/test.yml, and linting is configured via .golangci.yaml.
- The repo is therefore centered on a high-performance library core, with tests and benchmarks reinforcing the cache contracts and performance claims.
12. Main data flow to diagram
- Consumer calls root constructor → root wrapper selects cache flavor/options → keys are optionally routed by mux to a shard → shard delegates to a core cache instance → core performs lookup/update/eviction using its internal array-based structure and optional TTL clock.
Main entry points: tlru.go TLRU Core · mux.go Mux · lru.go LRU Core · slru.go SLRU Core
Components 17
Public API
User-facing constructors and cache variants at the module root
- LRU API · Single-instance LRU constructors and options
lru.go - SLRU API · Single-instance segmented LRU constructors and options
slru.go - TLRU API · Single-instance TTL-aware cache constructors and options
tlru.go - Pool API · Sharded pooled cache constructors
pool.go - TPool API · Sharded TTL-aware pooled cache constructors
tpool.go
Core Engine
Fixed-capacity cache implementations and shared runtime support
- LRU Core · Array-based fixed-capacity LRU implementation
core/lru.go - SLRU Core · Probationary/protected segmented LRU implementation
core/slru.go - TLRU Core · TTL-aware LRU implementation with lazy expiration
core/tlru.go - Sync · Shared synchronization and locking support
core/sync.go - Types · Shared core data structures and generic types
core/types.go
Sharding Router
Key-to-shard routing and hash strategy selection
Time
Clock abstraction used by TTL-aware caches
- Clock · Abstract time source for TTL calculations
clock/clock.go
Test Utilities
Shared scaffolding for tests and benchmarks
- Test Util · Shared deterministic data and setup helpers
internal/testutil/setup.go
Other components
- Consumer · External code using the cache library
Connections 25
- Consumer → LRU API: uses · Consumers create single-instance LRU caches
- Consumer → SLRU API: uses · Consumers create single-instance segmented caches
- Consumer → TLRU API: uses · Consumers create TTL-aware caches
- Consumer → Pool API: uses · Consumers create sharded pooled caches
- Consumer → TPool API: uses · Consumers create sharded TTL-aware pooled caches
- LRU API → LRU Core: wraps · Facade delegates to the core LRU engine
- SLRU API → SLRU Core: wraps · Facade delegates to the core SLRU engine
- TLRU API → TLRU Core: wraps · Facade delegates to the TTL-aware core engine
- Pool API → Mux: routes · Sharded caches use mux to select a shard
- TPool API → Mux: routes · TTL sharded caches use mux to select a shard
- Mux → FNV: hashes · Supports FNV-based shard distribution
- Mux → XXH: hashes · Supports XXH-based shard distribution
- Mux → MapHash: hashes · Supports maphash-based shard distribution
- Pool API → LRU Core: shards · Each shard owns a core cache instance
- TPool API → TLRU Core: shards · Each shard owns a TTL-aware core cache instance
- LRU Core → Sync: locks · Core cache state is protected by internal synchronization
- SLRU Core → Sync: locks · Core cache state is protected by internal synchronization
- TLRU Core → Sync: locks · Core cache state is protected by internal synchronization
- TLRU Core → Clock: reads time · TTL expiry checks use the clock abstraction
- LRU Core → Types: shares · Core engines share generic types and internal structures
- SLRU Core → Types: shares · Core engines share generic types and internal structures
- TLRU Core → Types: shares · Core engines share generic types and internal structures
- Test Util → LRU Core: supports tests · Shared test scaffolding exercises core cache behavior
- Test Util → SLRU Core: supports tests · Shared test scaffolding exercises core cache behavior
- Test Util → TLRU Core: supports tests · Shared test scaffolding exercises core cache behavior