# siddharthvaddem/openscreen architecture

> Architecture diagram of the GitHub repository siddharthvaddem/openscreen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/siddharthvaddem/openscreen
- Repository: https://github.com/siddharthvaddem/openscreen

## Overview

1. OpenScreen is an Electron desktop application with a clear split between the **main process**, the **renderer/UI**, and an optional **native capture layer**. The user-facing app lives in `src/`, while Electron orchestration and OS integration live in `electron/`.

2. The **renderer** (`src/main.tsx`, `src/App.tsx`) hosts the product UI: launch flow, recording setup, timeline-based editor, playback, export dialogs, settings, and help. Most editor behavior is implemented as React components and hooks under `src/components`, `src/hooks`, and `src/lib`.

3. The **Electron main process** (`electron/main.ts`) is the application boundary for windows, menus, global shortcuts, IPC wiring, and platform-specific startup behavior. `electron/preload.ts` exposes a controlled bridge into the renderer, while `electron/ipc/handlers.ts` and related files define the IPC contract.

4. There is a dedicated **native-bridge abstraction** under `electron/native-bridge/` that normalizes platform-specific recording capabilities into shared services. This layer exposes cursor, project, and system services, and it is where the app decides whether to use native macOS/Windows capture or fallback/browser-based behavior.

5. The capture architecture is split by platform. On **macOS**, native recording is implemented with ScreenCaptureKit helpers under `electron/native/screencapturekit/`. On **Windows**, native recording is handled by the `wgc-capture` C++ helper under `electron/native/wgc-capture/`, which uses Windows Graphics Capture plus related audio/webcam/cursor components.

6. The app’s **recording flow** starts in the renderer launch UI (`src/components/launch/*`), moves through hooks such as `useScreenRecorder`, `recorderHandle`, and native hooks under `src/native/hooks/`, then crosses into Electron/native capture. The output is a recording session model that later feeds editing and export. Linux is structurally different: the README indicates it uses the browser capture pipeline rather than the native helpers.

7. The **editor subsystem** is centered on `src/components/video-editor/`. It includes the timeline, playback controls, annotation overlays, crop/trim/speed controls, blur/background settings, shortcut dialogs, and project persistence. This is the main post-recording workspace and is designed around a single project state with many derived visual layers.

8. Video playback and visual composition are handled by reusable logic in `src/components/video-editor/videoPlayback/` and `src/lib/`. These modules compute cursor motion, zoom regions, focus/follow behavior, blur, overlays, and annotation placement. They act as the rendering math layer between project state and visible frames.

9. The **export pipeline** lives in `src/lib/exporter/` and is a major subsystem. It contains frame decoding, frame rendering, audio encoding, muxing, GIF/video exporters, and timestamped frame queues. This is where the editor timeline and media sources are transformed into final deliverables such as MP4 or GIF.

10. **Audio and captions** are separate processing flows. Audio level and waveform features use hooks like `useAudioPeaks` plus a worker. Captions are generated in `src/lib/captioning/`, including extraction, transcription, and annotation conversion. The README emphasizes that caption generation runs on-device/offline, so it is an internal processing pipeline rather than a cloud service integration.

11. **Cursor behavior** is a distinct architecture concern because the app supports cursor themes, click effects, smoothing, and native cursor recording on supported platforms. Shared cursor logic lives in `src/lib/cursor/`, while native cursor capture and telemetry are handled in the Electron/native-bridge layer. The architecture cleanly separates “record the cursor” from “render/edit the cursor during playback.”

12. **Persistence and preferences** are mostly local and project-scoped. `src/lib/projectPersistence.ts`, `userPreferences.ts`, and related editor defaults store configuration and project state, while `electron/native-bridge/store.ts` and service modules bridge to OS-level or app-level storage where needed.

13. The repository includes substantial **localization and cross-platform packaging** concerns. UI strings are organized under `src/i18n/locales/*`, with runtime loading/config in `src/i18n/`. Packaging and distribution are handled through Electron Builder, Nix flake/module files, and platform-specific scripts for macOS helpers, Windows helpers, and release automation.

14. Tests are layered around the same boundaries: unit tests in `src/` and `electron/`, browser/Vitest tests for exporter and playback logic, Playwright E2E tests under `tests/e2e/`, and helper scripts for validating native cursor/recording behavior. This reflects a codebase where the highest-risk areas are the capture pipeline and export pipeline rather than a conventional web backend.

Main entry points: [`electron/native-bridge`](https://github.com/siddharthvaddem/openscreen/tree/main/electron/native-bridge) (Native bridge), [`src/components/video-editor/VideoEditor.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoEditor.tsx) (Video editor), [`src/main.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/src/main.tsx) (App shell), [`src/lib/exporter`](https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/exporter) (Export pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_electron["Electron runtime"]
  node_n_main["Main process<br/>electron host<br/>[main.ts]"]
  node_n_preload["Preload bridge<br/>secure bridge<br/>[preload.ts]"]
  node_n_ipc["IPC handlers<br/>ipc contract<br/>[handlers.ts]"]
  node_n_nativebridge["Native bridge<br/>platform abstraction"]
  node_n_systemsvc["System service<br/>os service<br/>[systemService.ts]"]
  node_n_projectsvc["Project service<br/>storage service<br/>[projectService.ts]"]
  node_n_cursorsvc["Cursor service<br/>[cursorService.ts]"]
  node_n_store["Bridge store<br/>state bridge<br/>[store.ts]"]
end

subgraph group_g_renderer["Renderer UI"]
  node_n_app["App shell<br/>react entry<br/>[main.tsx]"]
  node_n_launch["Launch flow<br/>recording setup<br/>[LaunchWindow.tsx]"]
  node_n_editor["Video editor<br/>editor workspace<br/>[VideoEditor.tsx]"]
  node_n_timeline["Timeline<br/>[TimelineEditor.tsx]"]
  node_n_playback["Playback engine<br/>playback view<br/>[VideoPlayback.tsx]"]
end

subgraph group_g_shared["Shared logic"]
  node_n_export["Export pipeline<br/>media exporter"]
  node_n_caption["Captioning<br/>offline transcription"]
  node_n_audio["Audio analysis<br/>audio features<br/>[useAudioPeaks.ts]"]
  node_n_cursor["Cursor logic<br/>cursor rendering"]
  node_n_persist["Persistence<br/>project state"]
  node_n_i18n["Localization<br/>locale loader"]
  node_n_nativehooks["Native hooks<br/>renderer bridge hooks"]
  node_n_rendererlib["Playback math<br/>composition math"]
end

subgraph group_g_native["Native capture"]
  node_n_macos["macOS capture<br/>screen capture kit"]
  node_n_windows["Windows capture<br/>wgc helper"]
end

node_n_main -->|"exposes"| node_n_preload
node_n_main -->|"routes"| node_n_ipc
node_n_main -->|"starts"| node_n_nativebridge
node_n_preload -->|"bridges to"| node_n_app
node_n_ipc -->|"calls"| node_n_nativebridge
node_n_nativebridge -->|"provides"| node_n_systemsvc
node_n_nativebridge -->|"provides"| node_n_projectsvc
node_n_nativebridge -->|"provides"| node_n_cursorsvc
node_n_nativebridge -->|"persists"| node_n_store
node_n_nativebridge -->|"uses"| node_n_macos
node_n_nativebridge -->|"uses"| node_n_windows
node_n_app -->|"mounts"| node_n_launch
node_n_app -->|"mounts"| node_n_editor
node_n_launch -->|"starts recording"| node_n_nativehooks
node_n_nativehooks -->|"requests capture"| node_n_nativebridge
node_n_editor -->|"contains"| node_n_timeline
node_n_editor -->|"contains"| node_n_playback
node_n_editor -->|"exports"| node_n_export
node_n_editor -->|"saves"| node_n_persist
node_n_timeline -->|"computes"| node_n_rendererlib
node_n_playback -->|"renders with"| node_n_rendererlib
node_n_playback -->|"draws"| node_n_cursor
node_n_export -->|"reuses"| node_n_rendererlib
node_n_export -->|"consumes"| node_n_audio
node_n_caption -->|"feeds"| node_n_export
node_n_audio -->|"visualizes"| node_n_timeline
node_n_cursor -->|"ingests"| node_n_nativehooks
node_n_i18n -->|"localizes"| node_n_app
node_n_persist -->|"configures"| node_n_i18n

click node_n_main "https://github.com/siddharthvaddem/openscreen/blob/main/electron/main.ts"
click node_n_preload "https://github.com/siddharthvaddem/openscreen/blob/main/electron/preload.ts"
click node_n_ipc "https://github.com/siddharthvaddem/openscreen/blob/main/electron/ipc/handlers.ts"
click node_n_nativebridge "https://github.com/siddharthvaddem/openscreen/tree/main/electron/native-bridge"
click node_n_systemsvc "https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/services/systemService.ts"
click node_n_projectsvc "https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/services/projectService.ts"
click node_n_cursorsvc "https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/services/cursorService.ts"
click node_n_store "https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/store.ts"
click node_n_app "https://github.com/siddharthvaddem/openscreen/blob/main/src/main.tsx"
click node_n_launch "https://github.com/siddharthvaddem/openscreen/blob/main/src/components/launch/LaunchWindow.tsx"
click node_n_editor "https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoEditor.tsx"
click node_n_timeline "https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/timeline/TimelineEditor.tsx"
click node_n_playback "https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoPlayback.tsx"
click node_n_export "https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/exporter"
click node_n_caption "https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/captioning"
click node_n_audio "https://github.com/siddharthvaddem/openscreen/blob/main/src/hooks/useAudioPeaks.ts"
click node_n_cursor "https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/cursor"
click node_n_persist "https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/projectPersistence.ts"
click node_n_i18n "https://github.com/siddharthvaddem/openscreen/tree/main/src/i18n"
click node_n_nativehooks "https://github.com/siddharthvaddem/openscreen/tree/main/src/native/hooks"
click node_n_macos "https://github.com/siddharthvaddem/openscreen/tree/main/electron/native/screencapturekit"
click node_n_windows "https://github.com/siddharthvaddem/openscreen/tree/main/electron/native/wgc-capture"
click node_n_rendererlib "https://github.com/siddharthvaddem/openscreen/tree/main/src/components/video-editor/videoPlayback"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_main,node_n_preload,node_n_ipc,node_n_nativebridge,node_n_systemsvc,node_n_projectsvc,node_n_cursorsvc,node_n_store toneBlue
class node_n_app,node_n_launch,node_n_editor,node_n_timeline,node_n_playback toneAmber
class node_n_export,node_n_caption,node_n_audio,node_n_cursor,node_n_persist,node_n_i18n,node_n_nativehooks,node_n_rendererlib toneMint
class node_n_macos,node_n_windows toneRose
```

## Components

### Electron runtime

Main process, preload, IPC, and platform integration.

- [Main process](https://github.com/siddharthvaddem/openscreen/blob/main/electron/main.ts): electron host · Application boundary for windows, menus, shortcuts, IPC, and startup. · `electron/main.ts`
- [Preload bridge](https://github.com/siddharthvaddem/openscreen/blob/main/electron/preload.ts): secure bridge · Controlled API surface exposed from Electron into the renderer. · `electron/preload.ts`
- [IPC handlers](https://github.com/siddharthvaddem/openscreen/blob/main/electron/ipc/handlers.ts): ipc contract · Main-process handlers and recording stream wiring. · `electron/ipc/handlers.ts`
- [Native bridge](https://github.com/siddharthvaddem/openscreen/tree/main/electron/native-bridge): platform abstraction · Normalizes capture and OS services across platforms. · `electron/native-bridge`
- [System service](https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/services/systemService.ts): os service · System-level capabilities exposed through the bridge. · `electron/native-bridge/services/systemService.ts`
- [Project service](https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/services/projectService.ts): storage service · Project-oriented bridge for persistence and file operations. · `electron/native-bridge/services/projectService.ts`
- [Cursor service](https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/services/cursorService.ts): cursor service · Bridge for cursor capture and related telemetry. · `electron/native-bridge/services/cursorService.ts`
- [Bridge store](https://github.com/siddharthvaddem/openscreen/blob/main/electron/native-bridge/store.ts): state bridge · Shared bridge state and app-level storage coordination. · `electron/native-bridge/store.ts`

### Renderer UI

User-facing React app, launch flow, and editor workspace.

- [App shell](https://github.com/siddharthvaddem/openscreen/blob/main/src/main.tsx): react entry · Top-level renderer entry that mounts the product UI. · `src/main.tsx`
- [Launch flow](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/launch/LaunchWindow.tsx): recording setup · Source selection, countdown, and recording start UI. · `src/components/launch/LaunchWindow.tsx`
- [Video editor](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoEditor.tsx): editor workspace · Main post-recording workspace with project state and dialogs. · `src/components/video-editor/VideoEditor.tsx`
- [Timeline](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/timeline/TimelineEditor.tsx): editor timeline · Timeline editing, layers, rows, and keyframe navigation. · `src/components/video-editor/timeline/TimelineEditor.tsx`
- [Playback engine](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoPlayback.tsx): playback view · In-editor playback and composition view. · `src/components/video-editor/VideoPlayback.tsx`

### Shared logic

Reusable domain logic used by renderer, native bridge, and export flows.

- [Export pipeline](https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/exporter): media exporter · Frame decoding, rendering, audio encoding, muxing, and final video/GIF export. · `src/lib/exporter`
- [Captioning](https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/captioning): offline transcription · On-device audio extraction, transcription, and caption annotation conversion. · `src/lib/captioning`
- [Audio analysis](https://github.com/siddharthvaddem/openscreen/blob/main/src/hooks/useAudioPeaks.ts): audio features · Audio peaks, levels, and waveform support for the editor. · `src/hooks/useAudioPeaks.ts`
- [Cursor logic](https://github.com/siddharthvaddem/openscreen/tree/main/src/lib/cursor): cursor rendering · Cursor themes, smoothing, telemetry buffering, and playback rendering. · `src/lib/cursor`
- [Persistence](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/projectPersistence.ts): project state · Project persistence, defaults, and user preferences. · `src/components/video-editor/projectPersistence.ts`
- [Localization](https://github.com/siddharthvaddem/openscreen/tree/main/src/i18n): locale loader · Runtime i18n config and translated UI strings. · `src/i18n`
- [Native hooks](https://github.com/siddharthvaddem/openscreen/tree/main/src/native/hooks): renderer bridge hooks · Renderer-side hooks for native cursor and recording data. · `src/native/hooks`
- [Playback math](https://github.com/siddharthvaddem/openscreen/tree/main/src/components/video-editor/videoPlayback): composition math · Focus, zoom, overlay, cursor motion, and frame-composition utilities. · `src/components/video-editor/videoPlayback`

### Native capture

Platform-specific recording helpers and capture backends.

- [macOS capture](https://github.com/siddharthvaddem/openscreen/tree/main/electron/native/screencapturekit): screen capture kit · Native ScreenCaptureKit-based recording helpers. · `electron/native/screencapturekit`
- [Windows capture](https://github.com/siddharthvaddem/openscreen/tree/main/electron/native/wgc-capture): wgc helper · Windows Graphics Capture helper with audio, webcam, and cursor components. · `electron/native/wgc-capture`

## Connections

- Main process → Preload bridge: exposes · Main process publishes the preload bridge.
- Main process → IPC handlers: routes · Main process delegates renderer requests to IPC handlers.
- Main process → Native bridge: starts · Main process initializes the platform abstraction layer.
- Preload bridge → App shell: bridges to · Preload exposes controlled APIs to the renderer app.
- IPC handlers → Native bridge: calls · IPC handlers forward work into bridge services.
- Native bridge → System service: provides · Bridge exposes OS-level capabilities.
- Native bridge → Project service: provides · Bridge exposes project/file services.
- Native bridge → Cursor service: provides · Bridge exposes cursor-related services.
- Native bridge → Bridge store: persists · Bridge store holds shared bridge state.
- Native bridge → macOS capture: uses · macOS paths are delegated to native ScreenCaptureKit helpers.
- Native bridge → Windows capture: uses · Windows paths are delegated to the WGC helper.
- App shell → Launch flow: mounts · App shell presents launch flow before editing.
- App shell → Video editor: mounts · App shell switches into the editor workspace.
- Launch flow → Native hooks: starts recording · Launch flow triggers renderer-side native hooks.
- Native hooks → Native bridge: requests capture · Hooks cross into the Electron/native bridge for capture.
- Video editor → Timeline: contains · Editor centers on the timeline subsystem.
- Video editor → Playback engine: contains · Editor includes playback and composition view.
- Video editor → Export pipeline: exports · Editor sends project state into export pipeline.
- Video editor → Persistence: saves · Editor reads and writes project state and preferences.
- Timeline → Playback math: computes · Timeline uses composition math and playback utilities.
- Playback engine → Playback math: renders with · Playback view relies on shared render math.
- Playback engine → Cursor logic: draws · Playback uses cursor rendering logic and themes.
- Export pipeline → Playback math: reuses · Exporter shares rendering and overlay math.
- Export pipeline → Audio analysis: consumes · Exporter integrates audio features where needed.
- Captioning → Export pipeline: feeds · Caption output can be embedded into exports.
- Audio analysis → Timeline: visualizes · Audio peaks support waveform display on the timeline.
- Cursor logic → Native hooks: ingests · Cursor logic can consume native cursor data from hooks.
- Localization → App shell: localizes · Runtime locale loading feeds the renderer UI.
- Persistence → Localization: configures · Persistence and preferences include localized UI settings.