How ianahilario/playwright-notion-reporter works
1. This repository is a small TypeScript package, not an app: it plugs into Playwright Test as a custom reporter and writes one summary row per test run into a Notion database.
2. The runtime boundary is the Playwright test process. The reporter receives test lifecycle events from @playwright/test, aggregates results for the whole run, and only at the end sends a single write to Notion.
3. The main entrypoint is src/index.ts, which exposes the reporter implementation and the option types consumed from playwright.config.ts. This is the public API that downstream projects import via the package name.
4. The core data flow is: individual test attempts → reporter counts outcomes → final run summary → Notion database row. The summary includes passed, failed, skipped, flaky, total, and optional duration; “flaky” means a test that eventually passed after retries, and only the last attempt per test is counted.
5. Configuration is the main integration boundary. Consumers provide apiKey, databaseId, a required statusColumns mapping to exact Notion property names, and optional extra columns for metadata like branch or CI run URL. Empty custom values are omitted rather than written as blanks.
6. The Notion schema is intentionally strict and matters architecturally: numeric columns must exist for counts, Status must be a Notion Select property rather than a Status property, and select values like Passed/Failed must already exist in the database.
7. Error handling is centralized in src/notion-errors.ts. Instead of surfacing raw Notion JSON, the reporter classifies failures into actionable categories such as authentication issues, inaccessible databases, missing columns, wrong property types, and invalid select values, then prints fix guidance.
8. A key operational boundary is graceful degradation: if apiKey or databaseId is missing, the reporter logs a skip message and does not fail the test run. That keeps CI/test execution independent from Notion availability or secrets setup.
9. The repository includes unit tests under test/ for the counting logic and error categorization. These are likely the main verification layer for the reporter’s internal behavior without hitting Notion.
10. There is also an end-to-end example under example/ with Playwright tests (example/tests/demo.spec.ts) and a root playwright.config.ts. That sample exercises the full integration path: Playwright run → reporter → live Notion database.
11. Tooling is lightweight and standard for a library: TypeScript compilation (tsconfig.json), Vitest for unit tests (vitest.config.ts), and Playwright config for the example suite (playwright.config.ts / tsconfig.playwright.json). The package-lock and package.json indicate an npm-distributed module.
12. In diagram terms, the important components are: caller project’s Playwright runner, this reporter package, the Notion API as the external persistence service, and the local test/example harness used to validate the integration. The system has no internal server or database of its own; Notion is the only durable sink.
Main entry points: index.ts Reporter API · notion-errors.ts Error Categories · demo.spec.ts Demo Tests · vitest.config.ts Vitest
Components 15
Source
Published reporter implementation and shared runtime logic.
- Reporter API · Public entrypoint exported by the package for Playwright Test consumers, including reporter implementation and option types.
src/index.ts - Test Events · Consumes Playwright Test lifecycle events for a whole run and coordinates aggregation.
src/index.ts - Run Summary · Aggregates the final run counts: passed, failed, skipped, flaky, total, and optional duration.
src/index.ts - Config Schema · Reporter options boundary: apiKey, databaseId, statusColumns, and optional extra columns with omission of empty values.
src/index.ts - Notion Write · Sends one final database row to Notion at the end of the run.
src/index.ts - Error Categories · Classifies Notion failures into actionable categories with fix guidance instead of raw API payloads.
src/notion-errors.ts
Tests
Unit and example coverage for counting and error handling.
- Count Tests · Verifies outcome counting and retry/flaky semantics.
test/reporter.test.ts - Error Tests · Verifies Notion error categorization and guidance.
test/notion-errors.test.ts
Example
Playwright sample that exercises the full integration path.
- Demo Tests · Sample Playwright spec that exercises the reporter against a live Notion database.
example/tests/demo.spec.ts
Tooling
Compilation and test runners for the package.
- Vitest · Unit-test harness used for repository verification.
vitest.config.ts - Playwright Config · Root Playwright configuration for the sample integration path.
playwright.config.ts - TypeScript Build · TypeScript compilation setup for the published source.
tsconfig.json
External
Upstream test runner and downstream persistence service.
- Playwright Test · External runtime boundary that invokes the reporter from the consumer project's test process.
- Notion API · Remote database API used as the only durable sink for run summaries.
- npm Package · Published library consumed by downstream Playwright projects.
package.json
Connections 20
- npm Package → Reporter API: exports · The package publishes the reporter entrypoint and option types.
- Playwright Test → Reporter API: loads · Consumer Playwright processes load the reporter implementation.
- Playwright Test → Config Schema: provides · Projects pass reporter options from Playwright config.
- Reporter API → Test Events: handles · The reporter receives test lifecycle events from Playwright.
- Test Events → Run Summary: accumulates · Per-test attempts are reduced into a final run summary.
- Run Summary → Notion Write: summarizes · The final aggregate is transformed into a single Notion row.
- Config Schema → Notion Write: configures · Credentials, database id, status mappings, and extra columns drive the write.
- Config Schema → Run Summary: filters · Empty custom values are omitted from the final payload.
- Notion Write → Notion API: writes · A single database row is sent to Notion after the run ends.
- Notion Write → Error Categories: classifies · Failures are converted into actionable categories and guidance.
- Reporter API → Error Categories: uses · Reporter code relies on centralized Notion error handling.
- Count Tests → Run Summary: verifies · Unit tests cover counting and flaky retry behavior.
- Error Tests → Error Categories: verifies · Unit tests cover error classification logic.
- Demo Tests → Playwright Config: runs via · The sample suite is executed through the root Playwright config.
- Playwright Config → Reporter API: installs · The example config wires the reporter into Playwright.
- Demo Tests → Playwright Test: executes in · The demo spec runs inside Playwright Test.
- Reporter API → Notion API: optionally skips · Missing apiKey or databaseId causes a graceful skip instead of a failing run.
- Vitest → Count Tests: runs · Vitest executes the unit test suite.
- Vitest → Error Tests: runs · Vitest executes the error classification tests.
- TypeScript Build → Reporter API: builds · TypeScript compilation targets the published source.