g41797/tofu

1. What this repo is

Tofu is a Zig library for asynchronous, message-oriented networking. The core product is a reusable messaging runtime, not an application. Its architecture is centered on a message API, an event-driven reactor, and pluggable OS backends for stream transports like TCP and Unix Domain Sockets.

2. Public surface / domain model

The top-level library entry points are in src/tofu.zig, src/message.zig, src/address.zig, and src/status.zig. These define the concepts an application uses directly: messages, endpoints/addresses, and operation results/statuses. This is the boundary where application code stops and the runtime begins.

3. Async runtime core

The src/ampe subtree is the internal engine. It contains the reactor, poller, notifier, pools, queues, and channel management that move messages between user code and sockets without blocking. The important architectural idea is that “messages” are scheduled and transported by the runtime, not by application threads directly.

4. Reactor and event dispatch

src/ampe/Reactor.zig, poller.zig, Notifier.zig, channels.zig, and MchnGroup.zig form the control plane. The reactor owns the event loop; the poller waits for I/O readiness; notifiers wake the loop from other threads; channels and machine groups model connection endpoints and flow control. This is the central orchestration layer in the diagram.

5. Memory and buffering

src/ampe/Pool.zig, Appendable.zig, IntrusiveQueue.zig, and helper/internal files provide the runtime’s message and buffer management. The library explicitly includes internal pooling and appendable buffers, so the architecture should show a memory-management boundary inside the runtime rather than relying only on caller-owned allocations.

6. Transport abstraction

The src/platform tree is the portability boundary. It separates socket and poller behavior by platform family, so the core runtime can stay mostly OS-agnostic. This layer is where differences in readiness notification, socket creation, and trigger mechanisms are isolated.

7. Two platform strategies

There are two platform families: src/platform/stdposix and src/platform/posixnet. stdposix contains direct platform backends for Linux, macOS, and Windows (epoll, kqueue, wepoll). posixnet adds a wrapper layer with C adapters and FFI glue, suggesting an alternate portability path or a more specialized socket backend. A diagram should treat these as alternative backend stacks below the same core runtime.

8. OS-specific backends

Under src/platform/stdposix, each OS has its own socket and poller implementation: Linux uses epoll_backend.zig, macOS uses kqueue_backend.zig, and Windows uses wepoll_backend.zig, along with OS-specific socket creator and trigger files. These are the lowest-level components before the kernel boundary.

9. Data flow through the system

The main flow is: application code uses the public Tofu API, creates messages/addresses, and submits them into the ampe runtime; the reactor and poller route those messages to a channel/socket; the platform backend performs OS I/O; inbound data returns through the same path into message objects and callbacks/handlers. The system is duplex, so both directions use the same message-driven pipeline.

10. Concurrency and backpressure

The README emphasizes multithread safety and backpressure, and the code structure reflects that with notifier-based wakeups, queues, pools, and channel groups. That means the architecture is not “one thread per connection”; it is a shared async core that coordinates concurrent producers and consumers safely.

11. Usage patterns and examples

recipes/ contains cookbook-style examples such as MultiHomed.zig and service-oriented samples. These are not core runtime pieces, but they show how the public API is intended to be composed into higher-level messaging flows. For a diagram, they belong above the library layer as usage examples.

12. Tests as architecture validation

The tests/ tree is broad and maps closely to the subsystem boundaries: address, message, reactor, poller, sockets, platform-specific pollers, and tofu behavior. That makes the test suite a good indicator of the repo’s modular structure and the main seams where platform differences matter.

13. Build, docs, and delivery

build.zig and build.zig.zon define the Zig build; GitHub Actions run Linux, macOS, and Windows CI. The docs/ and docs_site/ trees contain generated documentation and site assets, while docs_site.sh / docs_zig.sh support doc generation. These are delivery/support systems around the library, not runtime components.

14. External boundaries

The only material external dependency boundary is the OS networking/event API layer: epoll, kqueue, and Windows I/O polling through backend adapters. Everything else is internal library structure. If you draw the architecture, the kernel boundary should sit directly beneath the platform backends, with the public API and examples clearly separated above the async core.