# frappe/pulse architecture

> Architecture diagram of the GitHub repository frappe/pulse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/frappe/pulse
- Repository: https://github.com/frappe/pulse

## Overview

1. **What this repo is**
   - `pulse` is a Frappe app that collects product telemetry and turns it into persisted `Pulse Event` records for analysis.
   - It sits at the boundary between event producers (browser and framework telemetry clients) and Frappe’s database-backed data model.

2. **Core ingest pipeline**
   - The primary flow is: client-side telemetry posts directly to Pulse → Pulse buffers events in Redis → a consumer drains Redis and writes `Pulse Event` rows in MariaDB/Frappe.
   - This separates high-volume ingest from durable storage and keeps the browser-facing request path lightweight.

3. **Client-side telemetry boundary**
   - `pulse/public/js/pulse_client.js` is the browser SDK served from the app’s asset pipeline.
   - The README indicates the browser loads it from `/assets/pulse/js/pulse_client.js` and posts events directly to the Pulse host, not through an intermediate framework relay.

4. **Server-side API surface**
   - `pulse/api/` is the likely HTTP entry point for ingest and related operations, with the README explicitly referencing `bulk_ingest`.
   - This API is the public boundary for event submission; it is where API key validation, CORS behavior, and request shaping matter.

5. **Redis-backed buffering layer**
   - The `redis_stream` doctype family (`Redis Stream`, `Redis Stream Consumer`, `Redis Stream Entry`) models the queueing/buffering layer in Frappe terms.
   - Architecturally, this is the “shock absorber” between transient incoming traffic and durable event storage.

6. **Persistence and domain model**
   - `Pulse Event` is the main fact table for analytics.
   - Supporting doctypes such as `Pulse Alias`, `Pulse Person`, `Pulse Log`, and `Pulse Settings` capture identity, metadata, operational logs, and configuration, respectively.

7. **Settings and enablement**
   - `Pulse Settings` controls runtime behavior, including API key matching and feature toggles referenced in the README.
   - The app also relies on site config values like `pulse_host`, `pulse_api_key`, and `allow_cors`, so deployment configuration is part of the architecture, not just app logic.

8. **Framework integration**
   - `pulse/hooks.py`, `pulse/constants.py`, and `pulse/patches/` indicate standard Frappe integration points for app lifecycle, configuration, and schema/data migrations.
   - The README’s mention of `pulse_force_enabled` shows there is an explicit test-only override path for local/e2e setups.

9. **E2E and test harness**
   - `pulse/e2e.py` provides operational helpers for test setup, draining queues, counting ingested rows, and cleanup.
   - End-to-end coverage spans browser client behavior and server ingest, which is important because the system’s correctness depends on the full pipeline, not just one service.

10. **Automation and external validation**
   - Cypress tests under `cypress/` validate browser-to-Pulse telemetry flow.
   - `pulse-k6.test.js` suggests load/performance validation exists for ingest behavior, and GitHub Actions (`.github/workflows/e2e.yml`) runs these checks in CI.

11. **Non-runtime/supporting pieces**
   - `pulse/utils/simulator.py` and the `tests/` directory support local simulation and unit-level verification.
   - These are useful for reproducing event flows without needing a full interactive product site.

12. **How to draw the architecture**
   - A good diagram would show: external app/browser → Pulse JS client or Frappe telemetry client → Pulse ingest API → Redis stream buffer → Redis consumer → Frappe/MariaDB `Pulse Event` storage.
   - Side boxes should include config/settings, identity/log doctypes, migration patches, and the CI/e2e harness that exercises the same flow end to end.

Main entry points: [`pulse/api`](https://github.com/frappe/pulse/tree/develop/pulse/api) (Ingest API), [`pulse/pulse/doctype/pulse_event/pulse_event.py`](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_event/pulse_event.py) (Pulse Event), [`pulse/pulse/doctype/redis_stream/redis_stream.py`](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/redis_stream/redis_stream.py) (Redis Stream), [`pulse/e2e.py`](https://github.com/frappe/pulse/blob/develop/pulse/e2e.py) (E2E Helpers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Telemetry clients"]
  node_browser_sdk["Browser SDK<br/>telemetry client<br/>[pulse_client.js]"]
end

subgraph group_core["Pulse app"]
  node_api_surface["Ingest API<br/>HTTP API"]
  node_buffer_layer[("Redis Stream<br/>queue model<br/>[redis_stream.py]")]
  node_stream_consumer["Stream Consumer<br/>queue worker"]
end

subgraph group_model["Domain model"]
  node_event_store[("Pulse Event<br/>fact doctype<br/>[pulse_event.py]")]
  node_settings["Pulse Settings<br/>config doctype<br/>[pulse_settings.py]"]
  node_person["Pulse Person<br/>identity doctype<br/>[pulse_person.py]"]
  node_alias["Pulse Alias<br/>identity doctype<br/>[pulse_alias.py]"]
  node_log["Pulse Log<br/>ops doctype<br/>[pulse_log.py]"]
end

subgraph group_ops["Lifecycle and ops"]
  node_anon["Auth Helpers<br/>request helper<br/>[anon.py]"]
  node_hooks["Hooks<br/>frappe integration<br/>[hooks.py]"]
  node_constants["Constants<br/>app config<br/>[constants.py]"]
  node_patches["Migrations<br/>schema patches"]
end

subgraph group_qa["Validation"]
  node_e2e_helpers["E2E Helpers<br/>test harness<br/>[e2e.py]"]
  node_cypress_tests["Browser Tests<br/>e2e suite<br/>[pulse_telemetry.js]"]
  node_load_test["Load Test<br/>performance test<br/>[pulse-k6.test.js]"]
  node_ci_workflow["CI Workflow<br/>github actions<br/>[e2e.yml]"]
  node_simulator["Simulator<br/>test utility<br/>[simulator.py]"]
end

node_browser_sdk -->|"posts events"| node_api_surface
node_api_surface -->|"validates"| node_anon
node_api_surface -->|"buffers"| node_buffer_layer
node_buffer_layer -->|"drains to"| node_stream_consumer
node_stream_consumer -->|"persists"| node_event_store
node_event_store -->|"joins identity"| node_person
node_event_store -->|"resolves aliases"| node_alias
node_event_store -->|"emits ops"| node_log
node_settings -->|"configures"| node_api_surface
node_constants -->|"shared rules"| node_api_surface
node_hooks -->|"registers"| node_api_surface
node_patches -->|"migrates"| node_event_store
node_patches -->|"migrates"| node_buffer_layer
node_e2e_helpers -->|"drains"| node_buffer_layer
node_e2e_helpers -->|"counts"| node_event_store
node_cypress_tests -->|"exercises"| node_browser_sdk
node_cypress_tests -->|"hits"| node_api_surface
node_load_test -->|"stresses"| node_api_surface
node_ci_workflow -->|"runs"| node_cypress_tests
node_ci_workflow -->|"runs"| node_load_test
node_simulator -->|"drives"| node_e2e_helpers

click node_browser_sdk "https://github.com/frappe/pulse/blob/develop/pulse/public/js/pulse_client.js"
click node_api_surface "https://github.com/frappe/pulse/tree/develop/pulse/api"
click node_buffer_layer "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/redis_stream/redis_stream.py"
click node_stream_consumer "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/redis_stream_consumer/redis_stream_consumer.py"
click node_event_store "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_event/pulse_event.py"
click node_settings "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_settings/pulse_settings.py"
click node_person "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_person/pulse_person.py"
click node_alias "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_alias/pulse_alias.py"
click node_log "https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_log/pulse_log.py"
click node_anon "https://github.com/frappe/pulse/blob/develop/pulse/anon.py"
click node_hooks "https://github.com/frappe/pulse/blob/develop/pulse/hooks.py"
click node_constants "https://github.com/frappe/pulse/blob/develop/pulse/constants.py"
click node_patches "https://github.com/frappe/pulse/tree/develop/pulse/patches"
click node_e2e_helpers "https://github.com/frappe/pulse/blob/develop/pulse/e2e.py"
click node_cypress_tests "https://github.com/frappe/pulse/blob/develop/cypress/integration/pulse_telemetry.js"
click node_load_test "https://github.com/frappe/pulse/blob/develop/pulse-k6.test.js"
click node_ci_workflow "https://github.com/frappe/pulse/blob/develop/.github/workflows/e2e.yml"
click node_simulator "https://github.com/frappe/pulse/blob/develop/pulse/utils/simulator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser_sdk toneBlue
class node_api_surface,node_buffer_layer,node_stream_consumer toneAmber
class node_event_store,node_settings,node_person,node_alias,node_log toneMint
class node_anon,node_hooks,node_constants,node_patches toneRose
class node_e2e_helpers,node_cypress_tests,node_load_test,node_ci_workflow,node_simulator toneIndigo
```

## Components

### Telemetry clients

Browser-side telemetry entry points and supporting helpers.

- [Browser SDK](https://github.com/frappe/pulse/blob/develop/pulse/public/js/pulse_client.js): telemetry client · Assets-served client that posts telemetry directly to Pulse. · `pulse/public/js/pulse_client.js`

### Pulse app

Server-side ingest, buffering, and persistence inside the Frappe app.

- [Ingest API](https://github.com/frappe/pulse/tree/develop/pulse/api): HTTP API · Public server boundary for bulk ingest and related operations. · `pulse/api`
- [Redis Stream](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/redis_stream/redis_stream.py): queue model · Frappe-backed buffering layer between ingest traffic and durable storage. · `pulse/pulse/doctype/redis_stream/redis_stream.py`
- [Stream Consumer](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/redis_stream_consumer/redis_stream_consumer.py): queue worker · Drains buffered events and converts them into persisted records. · `pulse/pulse/doctype/redis_stream_consumer/redis_stream_consumer.py`

### Domain model

Primary doctypes for events, identity, logs, and settings.

- [Pulse Event](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_event/pulse_event.py): fact doctype · Primary analytics record stored in MariaDB/Frappe. · `pulse/pulse/doctype/pulse_event/pulse_event.py`
- [Pulse Settings](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_settings/pulse_settings.py): config doctype · Runtime configuration and feature toggles for the app. · `pulse/pulse/doctype/pulse_settings/pulse_settings.py`
- [Pulse Person](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_person/pulse_person.py): identity doctype · Identity mapping for tracked users/persons. · `pulse/pulse/doctype/pulse_person/pulse_person.py`
- [Pulse Alias](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_alias/pulse_alias.py): identity doctype · Alias mapping used to connect identities across events. · `pulse/pulse/doctype/pulse_alias/pulse_alias.py`
- [Pulse Log](https://github.com/frappe/pulse/blob/develop/pulse/pulse/doctype/pulse_log/pulse_log.py): ops doctype · Operational log records for the telemetry system. · `pulse/pulse/doctype/pulse_log/pulse_log.py`

### Lifecycle and ops

Framework hooks, migrations, and auth/config helpers.

- [Auth Helpers](https://github.com/frappe/pulse/blob/develop/pulse/anon.py): request helper · Helpers for anonymous access and request-side validation. · `pulse/anon.py`
- [Hooks](https://github.com/frappe/pulse/blob/develop/pulse/hooks.py): frappe integration · App lifecycle and framework integration points. · `pulse/hooks.py`
- [Constants](https://github.com/frappe/pulse/blob/develop/pulse/constants.py): app config · Shared application constants and configuration values. · `pulse/constants.py`
- [Migrations](https://github.com/frappe/pulse/tree/develop/pulse/patches): schema patches · Data and schema migration scripts for deployed sites. · `pulse/patches`

### Validation

End-to-end, load, and CI coverage for the ingest pipeline.

- [E2E Helpers](https://github.com/frappe/pulse/blob/develop/pulse/e2e.py): test harness · Utilities for setup, queue draining, counting, and cleanup. · `pulse/e2e.py`
- [Browser Tests](https://github.com/frappe/pulse/blob/develop/cypress/integration/pulse_telemetry.js): e2e suite · Cypress coverage for browser-to-Pulse telemetry flow. · `cypress/integration/pulse_telemetry.js`
- [Load Test](https://github.com/frappe/pulse/blob/develop/pulse-k6.test.js): performance test · K6-based ingest and throughput validation. · `pulse-k6.test.js`
- [CI Workflow](https://github.com/frappe/pulse/blob/develop/.github/workflows/e2e.yml): github actions · Automation that runs end-to-end validation in GitHub Actions. · `.github/workflows/e2e.yml`
- [Simulator](https://github.com/frappe/pulse/blob/develop/pulse/utils/simulator.py): test utility · Local event-flow simulator for reproducing telemetry behavior. · `pulse/utils/simulator.py`

## Connections

- Browser SDK → Ingest API: posts events · Browser telemetry is sent directly to the Pulse ingest API.
- Ingest API → Auth Helpers: validates · API requests depend on request-side auth and allow/deny behavior.
- Ingest API → Redis Stream: buffers · Accepted events are written into Redis-backed stream storage.
- Redis Stream → Stream Consumer: drains to · Consumers read queued stream entries for durable processing.
- Stream Consumer → Pulse Event: persists · Consumed events are materialized as Pulse Event rows.
- Pulse Event → Pulse Person: joins identity · Event facts attach to person records for analysis.
- Pulse Event → Pulse Alias: resolves aliases · Identity aliases help unify related telemetry sources.
- Pulse Event → Pulse Log: emits ops · Operational activity is captured in Pulse Log records.
- Pulse Settings → Ingest API: configures · Settings influence ingest behavior and feature toggles.
- Constants → Ingest API: shared rules · Constants centralize app-wide values used by runtime code.
- Hooks → Ingest API: registers · Frappe hooks wire the app into the framework lifecycle.
- Migrations → Pulse Event: migrates · Migration scripts update stored telemetry data and schema.
- Migrations → Redis Stream: migrates · Patches also cover queue-related model transitions.
- E2E Helpers → Redis Stream: drains · Test helpers clear queued stream data during setup and teardown.
- E2E Helpers → Pulse Event: counts · Helpers verify the number of ingested durable rows.
- Browser Tests → Browser SDK: exercises · Browser tests validate the client-side telemetry path.
- Browser Tests → Ingest API: hits · Tests cover the server ingest boundary end to end.
- Load Test → Ingest API: stresses · Performance tests measure ingest behavior under load.
- CI Workflow → Browser Tests: runs · CI executes browser-to-server e2e validation.
- CI Workflow → Load Test: runs · CI also runs load/performance checks.
- Simulator → E2E Helpers: drives · The simulator supports reproducible ingest scenarios in tests.