How kxsystems/kdb-tick works
Project type and purpose
• tick.q – the “vanilla” tickerplant (TP) for ingesting and timestamping live data
• tick/r.q – the RDB (historical store) that subscribes to the TP and persists snapshots
• tick/u.q – the pub/sub script that client applications use to receive live updates
• Purpose: streamline market‑data acquisition (tickerplant), realtime dissemination (pub/sub), and storage (RDB).
File structure
• Top‐level:
• tick.q (script that launches the TP on port X)
• tick/ directory with:
– r.q (launches the RDB, usually on port Y)
– u.q (client subscription logic)
• No frontend/backend folders—this is a streaming‑data pipeline with distinct processes.
• The presence of only q scripts and no build/deploy configs indicates manual or kdb+ startup orchestration via shell scripts or system services.
README for additional insights
• Describes exactly how tick.q, r.q, and u.q interact and references kdb+ documentation.
a. Main Components
• Market Data Feed (external exchange or synthetic feed)
• Tickerplant process (tick.q)
• RDB process (tick/r.q)
• Subscriber processes (tick/u.q or other client apps)
• Underlying kdb+ instances (in‐memory tables, logs on disk)
b. Relationships and Interactions
• Market Data Feed pushes messages into the Tickerplant via TCP.
• The Tickerplant timestamps & logs every message to a local “log” file, then republishes it.
• RDB subscribes to the TP feed, writes rolling snapshots to disk (historical table), and optionally serves queries.
• Subscriber clients connect to the TP (and optionally to the RDB) to receive live tick updates.
c. Architectural Patterns
• Pub/Sub for real‐time distribution
• Write‐ahead logging in TP for recovery
• Snapshotting in RDB for persistence
• Separation of concerns: ingestion vs. persistence vs. consumption
d. Key Technologies
• kdb+ / q language
• TCP IPC between q processes
• File‐based logging (TP log files)
• Snapshot files on disk (RDB)
Elements
• Labels: “Market Data Feed,” “TP (tick.q),” “RDB (r.q),” “Subscriber (u.q).”
– Feed → TP → (RDB & Subscribers)
– RDB → Disk (snapshots)
– TP → Disk (log files)
– Ingestion (blue), Persistence (green), Consumption (orange)
– Rectangles for processes
• Annotations for ports and file paths, e.g., TP log file location, snapshot directory.