How cbschuld/valkey-pubsub works
Project Type and Purpose
• Type: A lightweight, TypeScript-native Pub/Sub library.
• Purpose: Provide broadcast-style messaging (publish/subscribe) on top of a Redis-compatible store (Valkey) and an in-memory queue for TypeScript apps (especially Mercurius GraphQL).
File Structure
• Top-level files:
• package.json, README.md, LICENSE, CI workflows → project metadata and pipelines.
• __tests__ → Jest tests validating core behaviors.
• src/:
• index.ts → entry point, exports ValkeyPubSub and PubSubGenericQueue.
• ValkeyPubSub.ts → implements Redis-backed pub/sub.
• PubSubGenericQueue.ts → in-memory queue for message buffering and delivery.
• Configs:
• TypeScript (tsconfig.*) and Jest (jest.config.mjs) → build/test tooling.
README for Architecture Clues
• Core components:
• ValkeyPubSub.create(config) → instantiates a Valkey (Redis) client and sets up connections.
• .publish({ topic, payload }) → stringifies and sends messages via Redis channels.
• .subscribe(topic, queue) → subscribes the Redis client to a channel and forwards incoming messages into a PubSubGenericQueue.
• PubSubGenericQueue → holds items in memory and invokes .onItem() callbacks for each new message.
• External integration:
• Designed to slot into Mercurius GraphQL for subscriptions.
Main Components (to include
a. Client Application
• E.g., a GraphQL server (Fastify + Mercurius) or any TypeScript process that uses this library.
b. ValkeyPubSub
• Manages one or more Redis connections (standalone or cluster).
• Implements publish, subscribe, and cleanup.
c. Valkey (Redis-Compatible Store)
• External service. Channels/topics live here.
• RESP2/RESP3 protocol.
d. PubSubGenericQueue
• In-memory queue; buffers messages and distributes to listeners via callback.
e. Listeners / Subscribers
• Application code consuming messages via queue.onItem().
Relationships and Data Flows
• Publish Flow:
1) Client calls ValkeyPubSub.publish({ topic, payload }).
2) ValkeyPubSub stringifies and pushes the message into Redis under that topic channel.
3) Redis broadcasts to all subscribers on that channel.
• Subscribe Flow:
1) Client creates a PubSubGenericQueue.
2) Client calls ValkeyPubSub.subscribe(topic, queue).
3) ValkeyPubSub issues a Redis SUBSCRIBE command for topic.
4) Upon receiving a message from Redis, ValkeyPubSub parses JSON and calls queue.push(message).
5) PubSubGenericQueue emits the message to all registered .onItem() callbacks.
• Cleanup:
• ValkeyPubSub.cleanup() closes Redis connections.
• queue.destroy() tears down in-memory resources.
Architectural Patterns and Design Principles
• Pub/Sub Messaging over Redis channels for decoupled, broadcast delivery.
• Queue Abstraction hides transport details from listeners, enabling in-memory delivery and buffering.
• Modular & Type-Safe: each piece is its own TS module, tests in Jest, can be imported as CJS or ESM.
• Extensibility: You could swap out the queue for another implementation or point to a different Redis client.
Essential Elements
• Footnote: Configuration object passed into ValkeyPubSub.create() (addresses, protocol, clusterMode).