How siddharthvaddem/openscreen works
1. OpenScreen is an Electron desktop application with a clear split between the main process, the renderer/UI, and an optional native capture layer. The user-facing app lives in src/, while Electron orchestration and OS integration live in electron/.
2. The renderer (src/main.tsx, src/App.tsx) hosts the product UI: launch flow, recording setup, timeline-based editor, playback, export dialogs, settings, and help. Most editor behavior is implemented as React components and hooks under src/components, src/hooks, and src/lib.
3. The Electron main process (electron/main.ts) is the application boundary for windows, menus, global shortcuts, IPC wiring, and platform-specific startup behavior. electron/preload.ts exposes a controlled bridge into the renderer, while electron/ipc/handlers.ts and related files define the IPC contract.
4. There is a dedicated native-bridge abstraction under electron/native-bridge/ that normalizes platform-specific recording capabilities into shared services. This layer exposes cursor, project, and system services, and it is where the app decides whether to use native macOS/Windows capture or fallback/browser-based behavior.
5. The capture architecture is split by platform. On macOS, native recording is implemented with ScreenCaptureKit helpers under electron/native/screencapturekit/. On Windows, native recording is handled by the wgc-capture C++ helper under electron/native/wgc-capture/, which uses Windows Graphics Capture plus related audio/webcam/cursor components.
6. The app’s recording flow starts in the renderer launch UI (src/components/launch/*), moves through hooks such as useScreenRecorder, recorderHandle, and native hooks under src/native/hooks/, then crosses into Electron/native capture. The output is a recording session model that later feeds editing and export. Linux is structurally different: the README indicates it uses the browser capture pipeline rather than the native helpers.
7. The editor subsystem is centered on src/components/video-editor/. It includes the timeline, playback controls, annotation overlays, crop/trim/speed controls, blur/background settings, shortcut dialogs, and project persistence. This is the main post-recording workspace and is designed around a single project state with many derived visual layers.
8. Video playback and visual composition are handled by reusable logic in src/components/video-editor/videoPlayback/ and src/lib/. These modules compute cursor motion, zoom regions, focus/follow behavior, blur, overlays, and annotation placement. They act as the rendering math layer between project state and visible frames.
9. The export pipeline lives in src/lib/exporter/ and is a major subsystem. It contains frame decoding, frame rendering, audio encoding, muxing, GIF/video exporters, and timestamped frame queues. This is where the editor timeline and media sources are transformed into final deliverables such as MP4 or GIF.
10. Audio and captions are separate processing flows. Audio level and waveform features use hooks like useAudioPeaks plus a worker. Captions are generated in src/lib/captioning/, including extraction, transcription, and annotation conversion. The README emphasizes that caption generation runs on-device/offline, so it is an internal processing pipeline rather than a cloud service integration.
11. Cursor behavior is a distinct architecture concern because the app supports cursor themes, click effects, smoothing, and native cursor recording on supported platforms. Shared cursor logic lives in src/lib/cursor/, while native cursor capture and telemetry are handled in the Electron/native-bridge layer. The architecture cleanly separates “record the cursor” from “render/edit the cursor during playback.”
12. Persistence and preferences are mostly local and project-scoped. src/lib/projectPersistence.ts, userPreferences.ts, and related editor defaults store configuration and project state, while electron/native-bridge/store.ts and service modules bridge to OS-level or app-level storage where needed.
13. The repository includes substantial localization and cross-platform packaging concerns. UI strings are organized under src/i18n/locales/*, with runtime loading/config in src/i18n/. Packaging and distribution are handled through Electron Builder, Nix flake/module files, and platform-specific scripts for macOS helpers, Windows helpers, and release automation.
14. Tests are layered around the same boundaries: unit tests in src/ and electron/, browser/Vitest tests for exporter and playback logic, Playwright E2E tests under tests/e2e/, and helper scripts for validating native cursor/recording behavior. This reflects a codebase where the highest-risk areas are the capture pipeline and export pipeline rather than a conventional web backend.
Main entry points: native-bridge Native bridge · VideoEditor.tsx Video editor · main.tsx App shell · exporter Export pipeline
Components 23
Electron runtime
Main process, preload, IPC, and platform integration.
- Main process · Application boundary for windows, menus, shortcuts, IPC, and startup.
electron/main.ts - Preload bridge · Controlled API surface exposed from Electron into the renderer.
electron/preload.ts - IPC handlers · Main-process handlers and recording stream wiring.
electron/ipc/handlers.ts - Native bridge · Normalizes capture and OS services across platforms.
electron/native-bridge - System service · System-level capabilities exposed through the bridge.
electron/native-bridge/services/systemService.ts - Project service · Project-oriented bridge for persistence and file operations.
electron/native-bridge/services/projectService.ts - Cursor service · Bridge for cursor capture and related telemetry.
electron/native-bridge/services/cursorService.ts - Bridge store · Shared bridge state and app-level storage coordination.
electron/native-bridge/store.ts
Renderer UI
User-facing React app, launch flow, and editor workspace.
- App shell · Top-level renderer entry that mounts the product UI.
src/main.tsx - Launch flow · Source selection, countdown, and recording start UI.
src/components/launch/LaunchWindow.tsx - Video editor · Main post-recording workspace with project state and dialogs.
src/components/video-editor/VideoEditor.tsx - Timeline · Timeline editing, layers, rows, and keyframe navigation.
src/components/video-editor/timeline/TimelineEditor.tsx - Playback engine · In-editor playback and composition view.
src/components/video-editor/VideoPlayback.tsx
Shared logic
Reusable domain logic used by renderer, native bridge, and export flows.
- Export pipeline · Frame decoding, rendering, audio encoding, muxing, and final video/GIF export.
src/lib/exporter - Captioning · On-device audio extraction, transcription, and caption annotation conversion.
src/lib/captioning - Audio analysis · Audio peaks, levels, and waveform support for the editor.
src/hooks/useAudioPeaks.ts - Cursor logic · Cursor themes, smoothing, telemetry buffering, and playback rendering.
src/lib/cursor - Persistence · Project persistence, defaults, and user preferences.
src/components/video-editor/projectPersistence.ts - Localization · Runtime i18n config and translated UI strings.
src/i18n - Native hooks · Renderer-side hooks for native cursor and recording data.
src/native/hooks - Playback math · Focus, zoom, overlay, cursor motion, and frame-composition utilities.
src/components/video-editor/videoPlayback
Native capture
Platform-specific recording helpers and capture backends.
- macOS capture · Native ScreenCaptureKit-based recording helpers.
electron/native/screencapturekit - Windows capture · Windows Graphics Capture helper with audio, webcam, and cursor components.
electron/native/wgc-capture
Connections 29
- Main process → Preload bridge: exposes · Main process publishes the preload bridge.
- Main process → IPC handlers: routes · Main process delegates renderer requests to IPC handlers.
- Main process → Native bridge: starts · Main process initializes the platform abstraction layer.
- Preload bridge → App shell: bridges to · Preload exposes controlled APIs to the renderer app.
- IPC handlers → Native bridge: calls · IPC handlers forward work into bridge services.
- Native bridge → System service: provides · Bridge exposes OS-level capabilities.
- Native bridge → Project service: provides · Bridge exposes project/file services.
- Native bridge → Cursor service: provides · Bridge exposes cursor-related services.
- Native bridge → Bridge store: persists · Bridge store holds shared bridge state.
- Native bridge → macOS capture: uses · macOS paths are delegated to native ScreenCaptureKit helpers.
- Native bridge → Windows capture: uses · Windows paths are delegated to the WGC helper.
- App shell → Launch flow: mounts · App shell presents launch flow before editing.
- App shell → Video editor: mounts · App shell switches into the editor workspace.
- Launch flow → Native hooks: starts recording · Launch flow triggers renderer-side native hooks.
- Native hooks → Native bridge: requests capture · Hooks cross into the Electron/native bridge for capture.
- Video editor → Timeline: contains · Editor centers on the timeline subsystem.
- Video editor → Playback engine: contains · Editor includes playback and composition view.
- Video editor → Export pipeline: exports · Editor sends project state into export pipeline.
- Video editor → Persistence: saves · Editor reads and writes project state and preferences.
- Timeline → Playback math: computes · Timeline uses composition math and playback utilities.
- Playback engine → Playback math: renders with · Playback view relies on shared render math.
- Playback engine → Cursor logic: draws · Playback uses cursor rendering logic and themes.
- Export pipeline → Playback math: reuses · Exporter shares rendering and overlay math.
- Export pipeline → Audio analysis: consumes · Exporter integrates audio features where needed.
- Captioning → Export pipeline: feeds · Caption output can be embedded into exports.
- Audio analysis → Timeline: visualizes · Audio peaks support waveform display on the timeline.
- Cursor logic → Native hooks: ingests · Cursor logic can consume native cursor data from hooks.
- Localization → App shell: localizes · Runtime locale loading feeds the renderer UI.
- Persistence → Localization: configures · Persistence and preferences include localized UI settings.