How rushi055/notifyhub works
1. Overall shape
- NotifyHub is a single Node.js codebase split into multiple independently runnable processes: an Express API server, a Kafka consumer, BullMQ workers, and a browser frontend.
- The architecture is event-driven: the API accepts requests quickly, persists state in PostgreSQL, then hands off delivery work asynchronously.
2. HTTP/API boundary
- src/server.js is the main runtime entrypoint for the backend.
- src/routes/ defines the API surface: auth, notifications, notify, and preferences.
- Requests are protected by src/middleware/auth.js and throttled by src/middleware/rateLimiter.js, so the API is the enforcement point for identity and per-user request volume.
3. Business logic layer
- src/services/ contains the core use-case logic:
- authService.js for registration/login and JWT-related flows
- notificationService.js for creation, persistence, dispatch, and status updates
- preferenceService.js for user delivery settings and caching behavior
- This layer is the main boundary between HTTP handlers and infrastructure dependencies.
4. Persistent storage
- PostgreSQL is the system of record.
- src/db/postgres.js is the database access layer.
- Migrations in src/db/migrations/ create three core tables:
- users
- notifications
- user_preferences
- Notification delivery state is persisted here, so delivery is durable even when downstream processing is delayed.
5. Async event pipeline
- src/kafka/producer.js, consumer.js, and client.js implement the Kafka boundary.
- The API publishes notification events to Kafka after inserting them into PostgreSQL.
- Kafka is the decoupling layer between request handling and delivery execution, allowing separate scaling of ingestion and processing.
6. Channel routing and job queues
- The Kafka consumer routes events into BullMQ queues defined in src/queues/.
- There are two delivery queues:
- emailQueue.js
- inappQueue.js
- This is the point where a single notification fan-outs into channel-specific work.
7. Delivery workers
- src/queues/emailWorker.js handles email delivery, using Nodemailer and user email data from the database.
- src/queues/inappWorker.js handles real-time in-app delivery.
- Workers are separate processes, so retries, concurrency, and failure handling are isolated from the API.
8. Redis usage
- src/redis/client.js supports low-latency shared state.
- Redis is used for preference caching, rate limiting counters, and WebSocket/session mapping for online users.
- It is not the primary data store; it supports fast coordination across processes.
9. Real-time WebSocket path
- src/websocket/socketHandler.js manages Socket.io connections and the live in-app notification channel.
- The in-app worker checks Redis for connection state, then emits to connected clients when possible.
- This means real-time delivery depends on a shared connection registry, not on the API process itself.
10. Authentication and preferences flow
- src/routes/auth.js and src/services/authService.js cover user signup/login.
- src/routes/preferences.js and src/services/preferenceService.js manage per-user channel toggles and notification delivery settings.
- Preference data is persisted in PostgreSQL and cached in Redis for fast lookup during dispatch.
11. Operations and observability
- src/dashboard/bullboard.js mounts Bull Board at /dashboard to inspect queue health, retries, and failures.
- docker-compose.yml provides the supporting infrastructure: PostgreSQL, Redis, Kafka, and Zookeeper.
- The repo is designed for local reproducibility and for running each backend component independently in production-like deployments.
12. Frontend and test surfaces
- frontend/ contains a browser-based UI (app.js, HTML pages, CSS) for login, sending notifications, viewing history, and monitoring WebSocket activity.
- public/test.html is a lightweight test page for WebSocket verification.
- These frontend assets are separate from the backend API and consume it over HTTP plus WebSocket connections.
13. End-to-end notification lifecycle
- A client posts to /api/notify.
- The API validates, rate-limits, and stores the notification as pending.
- The notification is published to Kafka.
- The consumer reads it, checks preferences, and enqueues email and/or in-app jobs.
- Workers deliver the notification and update status to delivered or failed.
- Online users receive the in-app event immediately; offline users can retrieve it later from PostgreSQL.
14. Architecture diagram boundaries to show
- Frontend/browser clients
- Express API and middleware
- PostgreSQL as durable storage
- Redis as shared low-latency state
- Kafka as the async event backbone
- BullMQ as channel-specific job processing
- Email worker and in-app/WebSocket worker as separate delivery subsystems
Main entry points: server.js API Server · services Services · consumer.js Consumer · inappWorker.js In-App Worker
Components 17
Clients
Browser and test entry points that consume the system.
- Frontend UI · Browser-based UI for auth, sending notifications, history, and monitoring.
frontend/app.js - Test Page · Lightweight page for WebSocket verification.
public/test.html
Backend
Express API, middleware, services, and supporting runtime entrypoints.
- API Server · Main backend entrypoint that hosts the HTTP API and mounts middleware/routes.
src/server.js - Routes · API surface for auth, notifications, notify, and preferences.
src/routes - Middleware · Authentication and rate limiting for request enforcement.
src/middleware - Services · Core business logic for auth, notifications, and preferences.
src/services - Bull Board · Queue inspection UI mounted for retries, failures, and health.
src/dashboard/bullboard.js
State
Durable and shared infrastructure used by backend processes.
- PostgreSQL · System of record for users, notifications, and preferences.
src/db/postgres.js - Redis · Low-latency state for preference caching, rate limiting, and socket presence.
src/redis/client.js
Async
Kafka, queues, and workers that process notification delivery.
- Kafka Client · Kafka boundary used to publish and consume notification events.
src/kafka/client.js - Producer · Publishes notification events after persistence.
src/kafka/producer.js - Consumer · Reads notification events and routes them into delivery work.
src/kafka/consumer.js - Email Queue · Channel-specific queue for email delivery jobs.
src/queues/emailQueue.js - In-App Queue · Channel-specific queue for in-app delivery jobs.
src/queues/inappQueue.js - Email Worker · Processes email jobs and sends notifications by email.
src/queues/emailWorker.js - In-App Worker · Processes real-time in-app jobs and emits to connected clients.
src/queues/inappWorker.js - Socket Handler · Manages live connections and the in-app notification channel.
src/websocket/socketHandler.js
Connections 22
- Frontend UI → API Server: HTTP API · Browser UI talks to the backend over HTTP.
- Frontend UI → Socket Handler: WebSocket · Frontend subscribes to live notification events.
- Test Page → Socket Handler: WebSocket test · Test page verifies socket connectivity.
- API Server → Middleware: enforces · Requests pass through auth and rate limiting.
- API Server → Routes: mounts · Server exposes the API routes.
- Routes → Services: calls · Route handlers delegate to business logic.
- Services → PostgreSQL: persists · Services read and write durable application state.
- Services → Redis: caches · Preferences and shared counters use Redis.
- Services → Producer: publishes · Notification creation is handed off as an event.
- Producer → Kafka Client: uses · Producer sends events through the Kafka boundary.
- Consumer → Kafka Client: subscribes · Consumer receives events from Kafka.
- Consumer → Email Queue: enqueues · Email delivery work is queued separately.
- Consumer → In-App Queue: enqueues · In-app delivery work is queued separately.
- Email Worker → Email Queue: consumes · Worker processes email jobs from the queue.
- Email Worker → PostgreSQL: reads users · Worker resolves recipient and delivery data.
- In-App Worker → In-App Queue: consumes · Worker processes in-app jobs from the queue.
- In-App Worker → Redis: checks presence · Worker uses shared connection state for online users.
- In-App Worker → Socket Handler: emits · Worker pushes live notifications to connected clients.
- Bull Board → Email Queue: inspects · Dashboard surfaces queue health and failures.
- Bull Board → In-App Queue: inspects · Dashboard surfaces queue health and failures.
- API Server → PostgreSQL: reads/writes · API persists notification and preference state.
- API Server → Redis: rate limits · Shared counters support throttling.