How foyer-rs/foyer works
Main Components
• Library Core (foyer): This is the central crate where the primary API is exposed. It integrates the various functionalities of hybrid caching.
• In-Memory Cache (foyer-memory): Responsible for managing the in-memory cache, including different eviction strategies (LRU, FIFO, LFU, etc.). It implements zero-copy cache abstractions and provides high-performance thread-safe access.
• Disk Storage Engine (foyer-storage): Handles persistent storage for caching, managing both “large” and “small” objects, compression support, and device I/O. It includes specialized modules (such as device management, indexer, compressor, and statistics).
• Common Utilities (foyer-common): Contains shared modules (e.g., for metrics, tracing, asynchronous utilities, and testing helpers) that are used by both memory and storage components.
• CLI and Benchmarking Tools (foyer-cli and foyer-bench): These crates provide command-line interfaces and benchmarking utilities to aid both users and developers.
• Examples and Scripts: Located under the “examples” and “scripts” directories, they demonstrate different configurations and integrations (like Prometheus, Grafana, and tail-based tracing integrations).
Relationships and Interactions
• At the top level, application code (or external users) calls the API provided by the main “foyer” crate.
• The main API delegates caching operations to either the in-memory cache component (from foyer-memory) or, in a hybrid setup, to the storage engine (from foyer-storage) based on configuration.
• The common utilities in foyer-common support both memory and storage by offering shared infrastructure (for example, for event tracing, metrics, and utilities for asynchronous operations).
• The CLI tool (foyer-cli) interacts with the core to provide a user-friendly entry point, and the benchmark suite (foyer-bench) is used for performance testing.
• External integrations, as showcased in examples and configuration files under “etc”, show how observability (Prometheus, Grafana) gets integrated into the system.
Architectural Patterns and Principles
• Modular Design: Each major function (in-memory caching, storage engine, utilities) is separated into its own crate. This separation promotes clean boundaries and easier maintenance.
• Hybrid Cache Architecture: The central concept of combining an in-memory cache with a secondary disk cache is the core design principle. The project separates policies and algorithms for eviction, admission, and recovery.
• Configurability & Extensibility: The project exposes plug-and-play caching algorithms and configurations (e.g., different eviction strategies, compression options, and runtime options) that allow customization for diverse workloads.
• Concurrency & Performance: With a focus on high concurrency and low latency (using thread-safe mechanisms and asynchronous runtimes), the design supports heavy workloads.
Detailed Focus Areas
• Separation of Concerns: Make it obvious that the in-memory caching logic and disk storage logic are decoupled components that communicate through the hybrid cache API.
• Data Flow: Represent how a cache “get” or “insert” operation flows from the user’s call through the API, then into the memory cache, and potentially falls back to the disk storage if not found.
• Extensibility: Indicate configurable hooks like customizable eviction policies, compression options (e.g., Lz4), and runtime options.
• External Observability: Show how the project integrates with external systems (e.g., Prometheus metrics paths, Grafana dashboards) provided in the “etc” directory.
• Tooling: Highlight supporting tools (CLI and Bench) that assist in monitoring, testing, and benchmarking the different subsystems.