How zone-eu/wildduck works
Project Type and Purpose
• WildDuck is a backend mail server with multiple protocol servers (IMAP, POP3, LMTP), an HTTP/REST API (for webmail and external integrations), background workers/tasks, and ACME certificate management.
• It stores data in a sharded & replicated MongoDB cluster, indexes messages with Elasticsearch, and uses Redis (with Lua scripts) for counters/rate-limiting.
• A minimal frontend (static webmail UI) is served from the public/ directory.
Data Stores & External Services
• MongoDB cluster (sharded + replicated) – primary data store for users, mailboxes, message metadata.
• GridFS (MongoDB) – attachment storage backend (lib/attachments/gridstore-storage.js).
• Elasticsearch – full-text search indexing of message bodies (lib/elasticsearch.js).
• Redis – counters, rate-limit storage, Lua scripts under lib/lua/*.lua.
• Optional external SMTP/MTA (e.g., Haraka or Zone-MTA) for sending outbound mail (set up via setup/ scripts).
• Nginx (or any reverse proxy) – TLS termination and routing to the API server.
Relationships & Data Flows
• Client → Protocol Servers: Users connect via IMAP/POP3 to read or manipulate mail; commands are processed by imap-core or POP3 handlers, which read/write MongoDB and push events.
• MTA → LMTP: Incoming mail is delivered to the LMTP server, which stores messages and triggers indexing.
• Client/Browser → HTTP API: Webmail UI or API clients call REST endpoints; they authenticate, then retrieve or manipulate mail via lib/api modules.
• Background Worker ↔ Data Stores: Periodic tasks pull data from MongoDB/ES, enforce quotas, process ACME updates, trigger webhook events.
• ACME Manager → ACME CA & File System: Requests certificates, stores them in certs/, and updates TLS configuration.
Architectural Patterns & Design Principles
• Modular Multi-process Node.js: Separate processes for each protocol and function, sharing code under lib/.
• Plugin Architecture: Extensible via config/plugins.toml and plugins/ directory for custom logic (e.g., Rspamd integration).
• Event-Driven: Handlers under lib/handlers/ respond to mail events (on-append, on-fetch, etc.).
• Configuration-Driven: All services configured via config/*.toml files—allows horizontal scaling and zero-downtime restarts.
• Containerized Deployment: Dockerfiles and docker-compose.yml orchestrate the server and dependencies.
Technologies & Frameworks
• Node.js (v14+)
• Express (or a minimal HTTP framework) for REST API
• Custom imap-core library for IMAP protocol handling
• MongoDB, GridFS, Elasticsearch, Redis
• Docker & Docker Compose
• Lua scripting for Redis counters
• OpenAPI/Swagger for API docs (docs/api/openapidocs.json)