How scylladb/seastar works
Overview:
Key Components to Include:
Core Framework Components
• Reactor/ Event Loop: Illustrate the central event loop (found in src/core/reactor.cc and related files) that drives the non-blocking execution.
• Scheduler & Concurrency Management: Show how seastar manages tasks, futures, coroutines, and scheduling (modules like src/core/smp.cc, src/core/thread.cc, src/core/future.cc).
• Memory & Resource Management: Indicate the custom allocator usage and memory management utilities (src/core/memory.cc, include/seastar/util).
Networking and Protocol Support
• Native TCP/IP Stack: Represent the framework's own userspace TCP/IP stack (referenced in README under “The Native TCP/IP Stack” and in src/net/*).
• HTTP and RPC Components: Include separate modules for HTTP (src/http and include/seastar/http) and RPC functionalities (src/rpc, include/seastar/rpc).
Build and Configuration Infrastructure
• Build System: Outline how CMake (with modes debug, release, dev, sanitize) is used to configure and build the entire project including submodules like DPDK. Emphasize the roles of top-level build scripts (configure.py, CMakeLists.txt in various directories).
Sample Applications and Demos
• Apps Directory: Break out the sample applications (httpd, memcached, rpc_tester, io_tester, etc.) that demonstrate how to utilize Seastar’s capabilities. Indicate that these serve as both demos and testing grounds.
• Demos: Similarly, include the demos (under demos/) which show standalone examples like echo servers, TCP demos, and coroutine demonstrations.
Testing and Documentation
• Testing Framework: Capture how unit tests (tests/unit) and performance tests (tests/perf) ensure system correctness.
• Documentation & Examples: Mention the extensive documentation (in doc/) and tutorials, which help guide developers to integrate and extend Seastar within their own applications.
• Use clear, labeled blocks to represent each major component (Reactor, Scheduler, I/O, Networking, etc.). Provide hierarchical detail starting from core library internals (src and include directories) then branch out to supporting systems (apps, demos, tests).
– App → Seastar application template (src/core/app-template.cc) → Event loop reactor (src/core/reactor.cc).
– Networking layers: HTTP components depend on the underlying TCP/IP stack and reactor.
• Design patterns or principles: For example, note the asynchronous programming model based on futures and continuations, and how sharded components (e.g., in src/core/sharded.cc) enable scalability on multi-core systems.
• Supplemental details:
– Configuration flow: How the configure.py script wraps around CMake.
– Build modes and their implications on performance and testing (highlight debug, release, dev, sanitize modes).
– Since Seastar is a library/framework rather than a typical full-stack application, concentrate on its internal architectural layers and APIs rather than a frontend/back-end separation.
– Focus on how the efficient event loop and non-blocking I/O architecture interlock with the networking protocols and RPC mechanisms.
– Emphasize extension points: illustrate where external applications or services plug into the framework (for example, user-written apps linking against the Seastar API via pkg-config and CMake integration).
• Event Loop/Reactor
• Scheduler & Concurrency Engine
• I/O and Resource Manager
2. Adjacent blocks for “Networking & Protocols” that branch from the core (TCP/IP stack, HTTP, RPC).
5. Component with relevant technologies (e.g., C++20/23 standard, use of futures, coroutine support).