How allegro/bigcache works
Project Type and Purpose
• BigCache is a high-performance, in-memory cache library for Go (requires Go 1.12+).
• It provides lock-sharded storage of entries in large byte slices to avoid GC pressure.
• It also offers an HTTP wrapper (in the server package) exposing cache operations over REST.
File Structure
• Top-level Go files (e.g., bigcache.go, config.go, shard.go) implement the core caching logic.
• Utility files (hash.go, encoding.go, iterator.go, stats.go, utils.go) support hashing, serialization, iteration, metrics, etc.
• queue/ implements the eviction queue used by each shard.
• server/ contains an HTTP server (server.go) with handlers (cache_handlers.go, stats_handler.go), plus middleware.
• Tests (*_test.go) and benchmarks live alongside production code.
Key Components
A. Configuration Module
• Config struct: shard count, life/clean windows, max-entry size, callbacks.
B. Cache Manager (BigCache struct in bigcache.go)
• Holds a slice of Shard instances.
• Exposes methods: Set, Get, Delete, Len, Reset, Iterator.
• Spins up background cleanup (ticker) per config.
C. Shard
• Contains:
• map[uint64]uint32 for hash→offset lookup
• large byte slices storing concatenated entries
• an eviction queue of entry timestamps/offsets
• sync.RWMutex to guard concurrent access
• Handles entry insertion, lookup, expiry marking.
D. Hashing Subsystem
• fnv.go: FNV-1a hash of keys → uint64 shard index and map key.
E. Eviction & Cleanup
• queue/bytes_queue.go: ring buffer / time-ordered slots
• Periodic clean job (driven by CleanWindow) that pops expired offsets and frees space.
F. Stats & Logging
• stats.go: collects metrics per shard (hits, misses, entries).
• logger.go: optional verbose output for allocations.
G. Iterator
• iterator.go: sequential scan over shards/entries for range operations.
H. HTTP Server (optional)
• server/server.go: HTTP router
• Handlers: GET/SET/DELETE for entries, GET for stats
• middleware.go: logging, request parsing
Component Relationships and Data Flow
• Client code calls bigcache.New(config) → instantiates Cache Manager → allocates shards with queues & byte buffers.
• Set(key, value):
1. Hash key → determine shard
2. Lock shard, append entry bytes to slice, update map, push timestamp into queue
3. Release lock
• Get(key):
1. Hash key → shard
2. RLock shard, lookup offset, read bytes, return value
3. Release lock
• Cleanup goroutine (per BigCache instance): wakes every CleanWindow, iterates shards, evicts expired offsets via their queues.
• If HTTP server is enabled:
• External HTTP client → router → appropriate handler → calls underlying Set/Get → returns HTTP response
Architectural Patterns and Principles
• Shard-based partitioning to maximize parallelism (lock striping).
• Zero-GC design: contiguous byte buffers + primitive map types avoid GC pressure.
• Separation of concerns: hashing, storage management, eviction, stats, and presentation (HTTP) are modularized.
• Configurable eviction callbacks for extensibility.