# g41797/tofu architecture

> Architecture diagram of the GitHub repository g41797/tofu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/g41797/tofu
- Repository: https://github.com/g41797/tofu

## Overview

1. **What this repo is**
   Tofu is a Zig library for asynchronous, message-oriented networking. The core product is a reusable messaging runtime, not an application. Its architecture is centered on a message API, an event-driven reactor, and pluggable OS backends for stream transports like TCP and Unix Domain Sockets.

2. **Public surface / domain model**
   The top-level library entry points are in `src/tofu.zig`, `src/message.zig`, `src/address.zig`, and `src/status.zig`. These define the concepts an application uses directly: messages, endpoints/addresses, and operation results/statuses. This is the boundary where application code stops and the runtime begins.

3. **Async runtime core**
   The `src/ampe` subtree is the internal engine. It contains the reactor, poller, notifier, pools, queues, and channel management that move messages between user code and sockets without blocking. The important architectural idea is that “messages” are scheduled and transported by the runtime, not by application threads directly.

4. **Reactor and event dispatch**
   `src/ampe/Reactor.zig`, `poller.zig`, `Notifier.zig`, `channels.zig`, and `MchnGroup.zig` form the control plane. The reactor owns the event loop; the poller waits for I/O readiness; notifiers wake the loop from other threads; channels and machine groups model connection endpoints and flow control. This is the central orchestration layer in the diagram.

5. **Memory and buffering**
   `src/ampe/Pool.zig`, `Appendable.zig`, `IntrusiveQueue.zig`, and helper/internal files provide the runtime’s message and buffer management. The library explicitly includes internal pooling and appendable buffers, so the architecture should show a memory-management boundary inside the runtime rather than relying only on caller-owned allocations.

6. **Transport abstraction**
   The `src/platform` tree is the portability boundary. It separates socket and poller behavior by platform family, so the core runtime can stay mostly OS-agnostic. This layer is where differences in readiness notification, socket creation, and trigger mechanisms are isolated.

7. **Two platform strategies**
   There are two platform families: `src/platform/stdposix` and `src/platform/posixnet`. `stdposix` contains direct platform backends for Linux, macOS, and Windows (`epoll`, `kqueue`, `wepoll`). `posixnet` adds a wrapper layer with C adapters and FFI glue, suggesting an alternate portability path or a more specialized socket backend. A diagram should treat these as alternative backend stacks below the same core runtime.

8. **OS-specific backends**
   Under `src/platform/stdposix`, each OS has its own socket and poller implementation: Linux uses `epoll_backend.zig`, macOS uses `kqueue_backend.zig`, and Windows uses `wepoll_backend.zig`, along with OS-specific socket creator and trigger files. These are the lowest-level components before the kernel boundary.

9. **Data flow through the system**
   The main flow is: application code uses the public Tofu API, creates messages/addresses, and submits them into the `ampe` runtime; the reactor and poller route those messages to a channel/socket; the platform backend performs OS I/O; inbound data returns through the same path into message objects and callbacks/handlers. The system is duplex, so both directions use the same message-driven pipeline.

10. **Concurrency and backpressure**
   The README emphasizes multithread safety and backpressure, and the code structure reflects that with notifier-based wakeups, queues, pools, and channel groups. That means the architecture is not “one thread per connection”; it is a shared async core that coordinates concurrent producers and consumers safely.

11. **Usage patterns and examples**
   `recipes/` contains cookbook-style examples such as `MultiHomed.zig` and service-oriented samples. These are not core runtime pieces, but they show how the public API is intended to be composed into higher-level messaging flows. For a diagram, they belong above the library layer as usage examples.

12. **Tests as architecture validation**
   The `tests/` tree is broad and maps closely to the subsystem boundaries: address, message, reactor, poller, sockets, platform-specific pollers, and tofu behavior. That makes the test suite a good indicator of the repo’s modular structure and the main seams where platform differences matter.

13. **Build, docs, and delivery**
   `build.zig` and `build.zig.zon` define the Zig build; GitHub Actions run Linux, macOS, and Windows CI. The `docs/` and `docs_site/` trees contain generated documentation and site assets, while `docs_site.sh` / `docs_zig.sh` support doc generation. These are delivery/support systems around the library, not runtime components.

14. **External boundaries**
   The only material external dependency boundary is the OS networking/event API layer: epoll, kqueue, and Windows I/O polling through backend adapters. Everything else is internal library structure. If you draw the architecture, the kernel boundary should sit directly beneath the platform backends, with the public API and examples clearly separated above the async core.

Main entry points: [`src/ampe.zig`](https://github.com/g41797/tofu/blob/main/src/ampe.zig) (AMPE), [`src/tofu.zig`](https://github.com/g41797/tofu/blob/main/src/tofu.zig) (Tofu API), [`src/platform/stdposix`](https://github.com/g41797/tofu/tree/main/src/platform/stdposix) (Std POSIX), [`tests/tofu_tests.zig`](https://github.com/g41797/tofu/blob/main/tests/tofu_tests.zig) (Tests)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_public["Public API"]
  node_tofu_api["Tofu API<br/>library entry<br/>[tofu.zig]"]
  node_message["Messages<br/>domain model<br/>[message.zig]"]
  node_address["Addresses<br/>domain model<br/>[address.zig]"]
  node_status["Statuses<br/>result model<br/>[status.zig]"]
end

subgraph group_runtime["Async Runtime"]
  node_ampe_root["AMPE<br/>runtime core<br/>[ampe.zig]"]
  node_reactor(("Reactor<br/>event loop<br/>[Reactor.zig]"))
  node_poller["Poller<br/>io wait<br/>[poller.zig]"]
  node_notifier(("Notifier<br/>wakeup<br/>[Notifier.zig]"))
  node_channels["Channels<br/>connection plane<br/>[channels.zig]"]
  node_mchn_group["Machine Groups<br/>coordination<br/>[MchnGroup.zig]"]
  node_pool[("Pool<br/>[Pool.zig]")]
  node_appendable["Appendable<br/>buffering<br/>[Appendable.zig]"]
  node_intrusive_queue["Intrusive Queue<br/>[IntrusiveQueue.zig]"]
end

subgraph group_platform["Platform Backends"]
  node_stdposix["Std POSIX<br/>platform family"]
  node_posixnet["Posix Net<br/>platform family"]
  node_linux_backend{{"Linux Backend<br/>epoll backend<br/>[epoll_backend.zig]"}}
  node_mac_backend{{"macOS Backend<br/>kqueue backend<br/>[kqueue_backend.zig]"}}
  node_windows_backend{{"Windows Backend<br/>wepoll backend<br/>[wepoll_backend.zig]"}}
  node_posix_wrapper["Wrapper Layer<br/>ffi adapter<br/>[ffi.zig]"]
end

subgraph group_delivery["Delivery"]
  node_recipes["Recipes<br/>examples<br/>[cookbook.zig]"]
  node_docs_site["Docs Site<br/>documentation<br/>[mkdocs.yml]"]
  node_build["Build<br/>[build.zig]"]
  node_tests["Tests<br/>validation suite<br/>[tofu_tests.zig]"]
end

node_tofu_api -->|"exports"| node_message
node_tofu_api -->|"exports"| node_address
node_tofu_api -->|"exports"| node_status
node_tofu_api -->|"uses"| node_ampe_root
node_ampe_root -->|"owns"| node_reactor
node_ampe_root -->|"drives"| node_poller
node_ampe_root -->|"wakes"| node_notifier
node_ampe_root -->|"routes"| node_channels
node_ampe_root -->|"coordinates"| node_mchn_group
node_ampe_root -->|"allocates"| node_pool
node_ampe_root -->|"buffers"| node_appendable
node_ampe_root -->|"queues"| node_intrusive_queue
node_reactor -->|"dispatches"| node_poller
node_reactor -->|"receives wakeups"| node_notifier
node_channels -->|"targets"| node_stdposix
node_stdposix -->|"includes"| node_linux_backend
node_stdposix -->|"includes"| node_mac_backend
node_stdposix -->|"includes"| node_windows_backend
node_posixnet -->|"wraps"| node_posix_wrapper
node_recipes -->|"demonstrates"| node_tofu_api
node_docs_site -->|"documents"| node_tofu_api
node_build -->|"builds"| node_tofu_api
node_tests -->|"verifies"| node_reactor
node_tests -->|"verifies"| node_poller
node_tests -->|"verifies"| node_channels
node_tests -->|"verifies"| node_stdposix

click node_tofu_api "https://github.com/g41797/tofu/blob/main/src/tofu.zig"
click node_message "https://github.com/g41797/tofu/blob/main/src/message.zig"
click node_address "https://github.com/g41797/tofu/blob/main/src/address.zig"
click node_status "https://github.com/g41797/tofu/blob/main/src/status.zig"
click node_ampe_root "https://github.com/g41797/tofu/blob/main/src/ampe.zig"
click node_reactor "https://github.com/g41797/tofu/blob/main/src/ampe/Reactor.zig"
click node_poller "https://github.com/g41797/tofu/blob/main/src/ampe/poller.zig"
click node_notifier "https://github.com/g41797/tofu/blob/main/src/ampe/Notifier.zig"
click node_channels "https://github.com/g41797/tofu/blob/main/src/ampe/channels.zig"
click node_mchn_group "https://github.com/g41797/tofu/blob/main/src/ampe/MchnGroup.zig"
click node_pool "https://github.com/g41797/tofu/blob/main/src/ampe/Pool.zig"
click node_appendable "https://github.com/g41797/tofu/blob/main/src/ampe/Appendable.zig"
click node_intrusive_queue "https://github.com/g41797/tofu/blob/main/src/ampe/IntrusiveQueue.zig"
click node_stdposix "https://github.com/g41797/tofu/tree/main/src/platform/stdposix"
click node_posixnet "https://github.com/g41797/tofu/tree/main/src/platform/posixnet"
click node_linux_backend "https://github.com/g41797/tofu/blob/main/src/platform/stdposix/linux/epoll_backend.zig"
click node_mac_backend "https://github.com/g41797/tofu/blob/main/src/platform/stdposix/mac/kqueue_backend.zig"
click node_windows_backend "https://github.com/g41797/tofu/blob/main/src/platform/stdposix/windows/wepoll_backend.zig"
click node_posix_wrapper "https://github.com/g41797/tofu/blob/main/src/platform/posixnet/wrapper/ffi.zig"
click node_recipes "https://github.com/g41797/tofu/blob/main/recipes/cookbook.zig"
click node_docs_site "https://github.com/g41797/tofu/blob/main/docs_site/mkdocs.yml"
click node_build "https://github.com/g41797/tofu/blob/main/build.zig"
click node_tests "https://github.com/g41797/tofu/blob/main/tests/tofu_tests.zig"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tofu_api,node_message,node_address,node_status toneBlue
class node_ampe_root,node_reactor,node_poller,node_notifier,node_channels,node_mchn_group,node_pool,node_appendable,node_intrusive_queue toneAmber
class node_stdposix,node_posixnet,node_linux_backend,node_mac_backend,node_windows_backend,node_posix_wrapper toneMint
class node_recipes,node_docs_site,node_build,node_tests toneRose
```

## Components

### Public API

Library entry points and user-facing concepts.

- [Tofu API](https://github.com/g41797/tofu/blob/main/src/tofu.zig): library entry · Top-level library entry point for applications using the messaging runtime. · `src/tofu.zig`
- [Messages](https://github.com/g41797/tofu/blob/main/src/message.zig): domain model · Message type and message-oriented API used by callers and the runtime. · `src/message.zig`
- [Addresses](https://github.com/g41797/tofu/blob/main/src/address.zig): domain model · Endpoint and address abstractions for stream transports. · `src/address.zig`
- [Statuses](https://github.com/g41797/tofu/blob/main/src/status.zig): result model · Operation outcomes and status types exposed to callers. · `src/status.zig`

### Async Runtime

Message scheduling, event dispatch, buffering, and connection management.

- [AMPE](https://github.com/g41797/tofu/blob/main/src/ampe.zig): runtime core · Async message-processing engine that coordinates reactors, channels, queues, and buffering. · `src/ampe.zig`
- [Reactor](https://github.com/g41797/tofu/blob/main/src/ampe/Reactor.zig): event loop · Owns the event loop and orchestrates readiness, wakeups, and dispatch. · `src/ampe/Reactor.zig`
- [Poller](https://github.com/g41797/tofu/blob/main/src/ampe/poller.zig): io wait · Waits for OS readiness events and feeds them back into the reactor. · `src/ampe/poller.zig`
- [Notifier](https://github.com/g41797/tofu/blob/main/src/ampe/Notifier.zig): wakeup · Cross-thread wakeup mechanism that interrupts the event loop safely. · `src/ampe/Notifier.zig`
- [Channels](https://github.com/g41797/tofu/blob/main/src/ampe/channels.zig): connection plane · Models endpoints, flow, and message transport between runtime machines and sockets. · `src/ampe/channels.zig`
- [Machine Groups](https://github.com/g41797/tofu/blob/main/src/ampe/MchnGroup.zig): coordination · Groups machines and coordinates shared runtime state and backpressure. · `src/ampe/MchnGroup.zig`
- [Pool](https://github.com/g41797/tofu/blob/main/src/ampe/Pool.zig): memory pool · Reusable allocation boundary for runtime objects and message storage. · `src/ampe/Pool.zig`
- [Appendable](https://github.com/g41797/tofu/blob/main/src/ampe/Appendable.zig): buffering · Growable buffer abstraction used by the runtime when assembling payloads. · `src/ampe/Appendable.zig`
- [Intrusive Queue](https://github.com/g41797/tofu/blob/main/src/ampe/IntrusiveQueue.zig): queue · Internal queue used to move work and messages through the async core. · `src/ampe/IntrusiveQueue.zig`

### Platform Backends

OS portability layers and kernel-facing adapters.

- [Std POSIX](https://github.com/g41797/tofu/tree/main/src/platform/stdposix): platform family · Primary portability layer for POSIX-like and Windows poller/socket backends. · `src/platform/stdposix`
- [Posix Net](https://github.com/g41797/tofu/tree/main/src/platform/posixnet): platform family · Alternate backend stack with C adapters and FFI glue. · `src/platform/posixnet`
- [Linux Backend](https://github.com/g41797/tofu/blob/main/src/platform/stdposix/linux/epoll_backend.zig): epoll backend · Linux-specific sockets, creators, triggers, and epoll integration. · `src/platform/stdposix/linux/epoll_backend.zig`
- [macOS Backend](https://github.com/g41797/tofu/blob/main/src/platform/stdposix/mac/kqueue_backend.zig): kqueue backend · macOS-specific sockets, creators, triggers, and kqueue integration. · `src/platform/stdposix/mac/kqueue_backend.zig`
- [Windows Backend](https://github.com/g41797/tofu/blob/main/src/platform/stdposix/windows/wepoll_backend.zig): wepoll backend · Windows-specific sockets, creators, triggers, and wepoll integration. · `src/platform/stdposix/windows/wepoll_backend.zig`
- [Wrapper Layer](https://github.com/g41797/tofu/blob/main/src/platform/posixnet/wrapper/ffi.zig): ffi adapter · C adapters, poll wrapper, and FFI boundary for the posixnet path. · `src/platform/posixnet/wrapper/ffi.zig`

### Delivery

Examples, docs, build, and validation surfaces around the library.

- [Recipes](https://github.com/g41797/tofu/blob/main/recipes/cookbook.zig): examples · Cookbook-style usage examples showing how the public API is composed. · `recipes/cookbook.zig`
- [Docs Site](https://github.com/g41797/tofu/blob/main/docs_site/mkdocs.yml): documentation · Generated docs and site content for library guidance and API reference. · `docs_site/mkdocs.yml`
- [Build](https://github.com/g41797/tofu/blob/main/build.zig): build system · Zig build entry and dependency manifest for building and packaging. · `build.zig`
- [Tests](https://github.com/g41797/tofu/blob/main/tests/tofu_tests.zig): validation suite · Subsystem-oriented tests covering address, message, reactor, poller, sockets, and platform behavior. · `tests/tofu_tests.zig`

## Connections

- Tofu API → Messages: exports · Public API exposes message concepts directly.
- Tofu API → Addresses: exports · Public API exposes endpoint/address types.
- Tofu API → Statuses: exports · Public API exposes operation statuses and results.
- Tofu API → AMPE: uses · Applications call into the async runtime through the top-level library.
- AMPE → Reactor: owns · Core runtime centers on the reactor control plane.
- AMPE → Poller: drives · Runtime delegates readiness waiting to the poller.
- AMPE → Notifier: wakes · Runtime uses notifiers for cross-thread wakeups.
- AMPE → Channels: routes · Runtime moves messages through channel abstractions.
- AMPE → Machine Groups: coordinates · Runtime organizes machines into groups for shared flow control.
- AMPE → Pool: allocates · Runtime uses internal pools for reusable memory management.
- AMPE → Appendable: buffers · Runtime builds payloads through appendable buffers.
- AMPE → Intrusive Queue: queues · Runtime schedules work through internal queues.
- Reactor → Poller: dispatches · Reactor depends on the poller for readiness events.
- Reactor → Notifier: receives wakeups · Notifier interrupts the reactor loop from other threads.
- Channels → Std POSIX: targets · Channel/socket operations flow into platform-specific backends.
- Std POSIX → Linux Backend: includes · Linux path uses epoll-based backend components.
- Std POSIX → macOS Backend: includes · macOS path uses kqueue-based backend components.
- Std POSIX → Windows Backend: includes · Windows path uses wepoll-based backend components.
- Posix Net → Wrapper Layer: wraps · Alternate backend path adds a C/FFI wrapper layer.
- Recipes → Tofu API: demonstrates · Examples show intended composition of the public API.
- Docs Site → Tofu API: documents · Site content explains the public surface and runtime model.
- Build → Tofu API: builds · Build system compiles the library and supporting artifacts.
- Tests → Reactor: verifies · Tests cover reactor behavior and orchestration.
- Tests → Poller: verifies · Tests cover poller behavior and platform readiness handling.
- Tests → Channels: verifies · Tests cover channel and socket flow management.
- Tests → Std POSIX: verifies · Tests exercise platform-specific poller and socket backends.