How tikv/yatp works
Project type and purpose
• This is a Rust library crate (“yatp”) that implements a generic, adaptive, responsive thread‐pool.
• It is meant to be embedded in other Rust projects to schedule and execute tasks (callbacks or futures) across a dynamically adjusted set of worker threads.
File structure
• Top‐level:
• Cargo.toml, src/, benches/, README.md, CI configs.
• src/
• lib.rs: public API entrypoint.
• pool/: core thread‐pool implementation
– builder.rs: pool configuration and construction
– runner.rs: main scheduler loop
– spawn.rs: task submission logic
– worker.rs: worker‐thread lifecycle
– tests.rs: unit and integration tests for pool
• queue/: pluggable scheduling queues
– single_level.rs: simple FIFO queue
– priority.rs: priority‐based queue
– multilevel.rs: multi‐level feedback queue (MLFQ)
– extras.rs: other helper functions
• task/: task‐abstraction layer
– callback.rs: synchronous callback tasks
– future.rs: async Future support
• metrics.rs: runtime instrumentation and metrics collection
• pool.rs, queue.rs, task.rs: re‐exports and high‐level facades
• benches/: performance benchmarks showing usage patterns
README for additional insights
• Emphasis on three principles: Adaptive, Responsive, Generic
• Highlights MLFQ for advanced scheduling and built‐in Future support
Elements
• Clear labels for: PoolBuilder, Scheduler, Worker, Queue types, Task types, Metrics.
– API calls → PoolBuilder → Scheduler → Queue → Worker
– Worker → Metrics reporting
– Scheduler ↔ Worker (monitoring & scaling)