frappe/pulse

1. What this repo is

- pulse is a Frappe app that collects product telemetry and turns it into persisted Pulse Event records for analysis.

- It sits at the boundary between event producers (browser and framework telemetry clients) and Frappe’s database-backed data model.

2. Core ingest pipeline

- The primary flow is: client-side telemetry posts directly to Pulse → Pulse buffers events in Redis → a consumer drains Redis and writes Pulse Event rows in MariaDB/Frappe.

- This separates high-volume ingest from durable storage and keeps the browser-facing request path lightweight.

3. Client-side telemetry boundary

- pulse/public/js/pulse_client.js is the browser SDK served from the app’s asset pipeline.

- The README indicates the browser loads it from /assets/pulse/js/pulse_client.js and posts events directly to the Pulse host, not through an intermediate framework relay.

4. Server-side API surface

- pulse/api/ is the likely HTTP entry point for ingest and related operations, with the README explicitly referencing bulk_ingest.

- This API is the public boundary for event submission; it is where API key validation, CORS behavior, and request shaping matter.

5. Redis-backed buffering layer

- The redis_stream doctype family (Redis Stream, Redis Stream Consumer, Redis Stream Entry) models the queueing/buffering layer in Frappe terms.

- Architecturally, this is the “shock absorber” between transient incoming traffic and durable event storage.

6. Persistence and domain model

- Pulse Event is the main fact table for analytics.

- Supporting doctypes such as Pulse Alias, Pulse Person, Pulse Log, and Pulse Settings capture identity, metadata, operational logs, and configuration, respectively.

7. Settings and enablement

- Pulse Settings controls runtime behavior, including API key matching and feature toggles referenced in the README.

- The app also relies on site config values like pulse_host, pulse_api_key, and allow_cors, so deployment configuration is part of the architecture, not just app logic.

8. Framework integration

- pulse/hooks.py, pulse/constants.py, and pulse/patches/ indicate standard Frappe integration points for app lifecycle, configuration, and schema/data migrations.

- The README’s mention of pulse_force_enabled shows there is an explicit test-only override path for local/e2e setups.

9. E2E and test harness

- pulse/e2e.py provides operational helpers for test setup, draining queues, counting ingested rows, and cleanup.

- End-to-end coverage spans browser client behavior and server ingest, which is important because the system’s correctness depends on the full pipeline, not just one service.

10. Automation and external validation

- Cypress tests under cypress/ validate browser-to-Pulse telemetry flow.

- pulse-k6.test.js suggests load/performance validation exists for ingest behavior, and GitHub Actions (.github/workflows/e2e.yml) runs these checks in CI.

11. Non-runtime/supporting pieces

- pulse/utils/simulator.py and the tests/ directory support local simulation and unit-level verification.

- These are useful for reproducing event flows without needing a full interactive product site.

12. How to draw the architecture

- A good diagram would show: external app/browser → Pulse JS client or Frappe telemetry client → Pulse ingest API → Redis stream buffer → Redis consumer → Frappe/MariaDB Pulse Event storage.

- Side boxes should include config/settings, identity/log doctypes, migration patches, and the CI/e2e harness that exercises the same flow end to end.