# wavii/darner architecture

> Architecture diagram of the GitHub repository wavii/darner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wavii/darner
- Repository: https://github.com/wavii/darner

## Overview

##### Main components
- Client Interface – Represents the external clients that send requests to the queue server using the memcache protocol (the same protocol used by Kestrel).
- Network Server – This component is implemented in the “src/net” and “include/darner/net” directories. It accepts incoming connections on port 22133 (as described in the README), manages network I/O, and processes commands coming in from clients.
- Queue Manager – Located in “src/queue” and “include/darner/queue,” this component implements the core FIFO queue functionality including the two-phase reliable fetch where messages are processed reliably.
- Logging & Utilities – Distributed across “src/util” and “include/darner/util,” this module handles logging, stats, and other auxiliary functions used throughout the server.
- Build & Deployment – Comprising files like CMakeLists.txt, the “bin/make_dist.sh” script, and related documentation in “docs”, this represents how the system is built, tested, and deployed.
- Benchmarking and Tests – The “bench” and “tests” directories include performance benchmarks and test cases that validate behavior. These aren’t part of the runtime architecture but are essential for ensuring how the components interact under load.

##### Relationships and interactions
- Clients communicate with the Network Server via the memcache protocol.
- The Network Server parses client requests and delegates them to the Queue Manager.
- The Queue Manager processes queue operations (push, fetch, delete, etc.) and coordinates with the underlying persistence layer (which leverages external libraries like LevelDB and Snappy for on-disk storage, as per the README).
- Utility modules such as logging and stats capture operational information, feeding back to the server for diagnostics.
- Benchmark tests and scripts are used to evaluate throughput and latency by simulating client operations.

##### Important architectural patterns or design principles
- Separation of Concerns – The modular structure (net, queue, util) makes clear separation between the network layer and the queue management logic.
- Out-of-Process Data Storage – Instead of holding data in RAM, Darner is designed to handle message persistence externally, integrating with disk-based storage systems (leveraging log-structured storage concepts).
- Simplicity – The “No talking! Shhh!” approach emphasizes minimal configuration and feature set for high performance and low overhead.

##### Relevant technologies and frameworks
- CMake – For building and managing the project.
- Boost (and potentially other C++ libraries) – Likely used to support concurrency or advanced data structures although not explicitly seen, it is required by the README.
- LevelDB and Snappy – External dependencies to handle on-disk storage with compression/fast access.
- POSIX Sockets or equivalent – Underlying network communication in the “net” modules.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Runtime Components
    subgraph "Runtime Components"
        CI["Client Interface"]:::client
        NS["Network Server"]:::net
        QM["Queue Manager"]:::queue
        LU["Logging & Utilities"]:::util
        PL["Persistence Layer (LevelDB & Snappy)"]:::external
    end

    %% Development Tools
    subgraph "Development Tools"
        BD["Build & Deployment"]:::build
        BT["Benchmarking & Tests"]:::dev
    end

    %% Data Flow Connections
    CI -->|"memcache"| NS
    NS -->|"parses_commands"| QM
    QM -->|"persists_messages"| PL
    NS -- "logs/stats" --> LU
    QM -- "logs/stats" --> LU
    %% Optional: show that development tools are used for validation/release
    BD --- BT

    %% Click Events for Network Server
    click NS "https://github.com/wavii/darner/blob/master/include/darner/net/server.hpp"
    click NS "https://github.com/wavii/darner/blob/master/src/net/handler.cpp"
    click NS "https://github.com/wavii/darner/blob/master/src/net/request.cpp"

    %% Click Events for Queue Manager
    click QM "https://github.com/wavii/darner/blob/master/include/darner/queue/queue.h"
    click QM "https://github.com/wavii/darner/blob/master/src/queue/queue.cpp"

    %% Click Events for Logging & Utilities
    click LU "https://github.com/wavii/darner/blob/master/include/darner/util/log.h"
    click LU "https://github.com/wavii/darner/blob/master/src/util/log.cpp"

    %% Click Events for Build & Deployment
    click BD "https://github.com/wavii/darner/blob/master/CMakeLists.txt"
    click BD "https://github.com/wavii/darner/blob/master/bin/make_dist.sh"
    click BD "https://github.com/wavii/darner/blob/master/docs/benchmarks.md"

    %% Click Events for Benchmarking and Tests
    click BT "https://github.com/wavii/darner/blob/master/bench/db.cpp"
    click BT "https://github.com/wavii/darner/blob/master/tests/main.cpp"

    %% Styles
    classDef client fill:#a4d7f5,stroke:#333,stroke-width:2px;
    classDef net fill:#cfe2f3,stroke:#333,stroke-width:2px;
    classDef queue fill:#f4cccc,stroke:#333,stroke-width:2px;
    classDef util fill:#ead1dc,stroke:#333,stroke-width:2px;
    classDef external fill:#fff2cc,stroke:#333,stroke-width:2px;
    classDef build fill:#d9ead3,stroke:#333,stroke-width:2px;
    classDef dev fill:#d9d2e9,stroke:#333,stroke-width:2px;
```
