# zone-eu/wildduck architecture

> Architecture diagram of the GitHub repository zone-eu/wildduck, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zone-eu/wildduck
- Repository: https://github.com/zone-eu/wildduck

## Overview

##### Project Type and Purpose
- WildDuck is a **backend mail server** with multiple protocol servers (IMAP, POP3, LMTP), an HTTP/REST API (for webmail and external integrations), background workers/tasks, and ACME certificate management.
- It stores data in a **sharded & replicated MongoDB** cluster, indexes messages with **Elasticsearch**, and uses **Redis** (with Lua scripts) for counters/rate-limiting.
- A minimal frontend (static webmail UI) is served from the `public/` directory.

##### Data Stores & External Services
- MongoDB cluster (sharded + replicated) – primary data store for users, mailboxes, message metadata.
- GridFS (MongoDB) – attachment storage backend (`lib/attachments/gridstore-storage.js`).
- Elasticsearch – full-text search indexing of message bodies (`lib/elasticsearch.js`).
- Redis – counters, rate-limit storage, Lua scripts under `lib/lua/*.lua`.
- Optional external SMTP/MTA (e.g., Haraka or Zone-MTA) for sending outbound mail (set up via `setup/` scripts).
- Nginx (or any reverse proxy) – TLS termination and routing to the API server.

##### Relationships & Data Flows
- **Client → Protocol Servers**: Users connect via IMAP/POP3 to read or manipulate mail; commands are processed by `imap-core` or POP3 handlers, which read/write MongoDB and push events.
- **MTA → LMTP**: Incoming mail is delivered to the LMTP server, which stores messages and triggers indexing.
- **Client/Browser → HTTP API**: Webmail UI or API clients call REST endpoints; they authenticate, then retrieve or manipulate mail via `lib/api` modules.
- **Background Worker ↔ Data Stores**: Periodic tasks pull data from MongoDB/ES, enforce quotas, process ACME updates, trigger webhook events.
- **ACME Manager → ACME CA & File System**: Requests certificates, stores them in `certs/`, and updates TLS configuration.

##### Architectural Patterns & Design Principles
- **Modular Multi-process Node.js**: Separate processes for each protocol and function, sharing code under `lib/`.
- **Plugin Architecture**: Extensible via `config/plugins.toml` and `plugins/` directory for custom logic (e.g., Rspamd integration).
- **Event-Driven**: Handlers under `lib/handlers/` respond to mail events (`on-append`, `on-fetch`, etc.).
- **Configuration-Driven**: All services configured via `config/*.toml` files—allows horizontal scaling and zero-downtime restarts.
- **Containerized Deployment**: Dockerfiles and `docker-compose.yml` orchestrate the server and dependencies.

##### Technologies & Frameworks
- Node.js (v14+)
- Express (or a minimal HTTP framework) for REST API
- Custom `imap-core` library for IMAP protocol handling
- MongoDB, GridFS, Elasticsearch, Redis
- Docker & Docker Compose
- Lua scripting for Redis counters
- OpenAPI/Swagger for API docs (`docs/api/openapidocs.json`)

## Diagram (Mermaid)

```mermaid
flowchart TD
    client["Client (Mail Client)"]:::external
    browser["Browser (Webmail UI)"]:::external
    mta["External MTA"]:::external
    acmeCA["ACME CA"]:::external
    nginx["Nginx (TLS Termination)"]:::external

    subgraph "WildDuck Container"
        direction TB
        IMAP["IMAP Server"]:::protocol
        POP3["POP3 Server"]:::protocol
        LMTP["LMTP Server"]:::protocol
        API["HTTP API Server"]:::api
        UI["Static Webmail UI"]:::api
        Worker["Background Worker"]:::worker
        ACME["ACME Manager"]:::worker
        Webhooks["Webhooks Processor"]:::worker
        Plugins["Plugin System"]:::external
    end

    subgraph "Datastores & Services"
        direction TB
        MongoDB["MongoDB Cluster"]:::datastore
        ES["Elasticsearch"]:::datastore
        Redis["Redis"]:::datastore
        GridFS["GridFS Storage"]:::datastore
    end

    client -->|IMAP4rev1| IMAP
    client -->|POP3| POP3
    mta -->|LMTP| LMTP
    browser -->|HTTP/REST JSON| nginx
    nginx -->|proxy HTTP| API
    API -->|serves Static| UI
    IMAP -->|MongoDB ops| MongoDB
    IMAP -->|ES indexing| ES
    POP3 -->|MongoDB ops| MongoDB
    LMTP -->|store messages| MongoDB
    LMTP -->|ES index| ES
    API -->|MongoDB ops| MongoDB
    API -->|Redis counters| Redis
    API -->|plugin hooks| Plugins
    Worker -->|MongoDB ops| MongoDB
    Worker -->|ES index| ES
    Worker -->|rate-limit| Redis
    Worker -->|trigger webhooks| Webhooks
    Worker -->|plugin hooks| Plugins
    ACME -->|ACME requests| acmeCA
    ACME -->|store certs| nginx
    GridFS -.-> MongoDB
    MongoDB -->|attachments| GridFS

    click IMAP "https://github.com/zone-eu/wildduck/blob/master/imap.js"
    click POP3 "https://github.com/zone-eu/wildduck/blob/master/pop3.js"
    click LMTP "https://github.com/zone-eu/wildduck/blob/master/lmtp.js"
    click API "https://github.com/zone-eu/wildduck/blob/master/api.js"
    click UI "https://github.com/zone-eu/wildduck/tree/master/public/public/"
    click Worker "https://github.com/zone-eu/wildduck/blob/master/worker.js"
    click ACME "https://github.com/zone-eu/wildduck/blob/master/acme.js"
    click Webhooks "https://github.com/zone-eu/wildduck/blob/master/webhooks.js"
    click MongoDB "https://github.com/zone-eu/wildduck/blob/master/lib/db.js"
    click GridFS "https://github.com/zone-eu/wildduck/blob/master/lib/attachments/gridstore-storage.js"
    click ES "https://github.com/zone-eu/wildduck/blob/master/lib/elasticsearch.js"
    click Redis "https://github.com/zone-eu/wildduck/tree/master/lib/lua/"
    click Plugins "https://github.com/zone-eu/wildduck/blob/master/config/plugins.toml"
    click DockerCompose "https://github.com/zone-eu/wildduck/blob/master/docker-compose.yml"
    click Dockerfile "https://github.com/zone-eu/wildduck/tree/master/Dockerfile"
    click Nginx "https://github.com/zone-eu/wildduck/blob/master/setup/11_install_nginx.sh"

    classDef protocol fill:#a2fca2,stroke:#333
    classDef api fill:#a2d4fc,stroke:#333
    classDef worker fill:#fccfa2,stroke:#333
    classDef datastore fill:#d3d3d3,stroke:#333
    classDef external fill:#e0c0fc,stroke:#333
```
