How familiar-software/familiar works
1. Product shape and runtime
- Familiar is an Electron desktop app with a Node.js main process and a React-based dashboard renderer.
- Its core job is to sample the user’s screen and clipboard every few seconds, turn that into markdown, redact sensitive data, and store it locally for downstream AI tools.
2. Main-process orchestration
- src/main.js is the top-level entry point that wires together app startup, windows, tray/menu, updates, settings, capture, and IPC.
- The main process owns the long-lived system integrations: screen capture, filesystem writes, permissions, notifications/toasts, and app lifecycle.
3. Renderer boundary: dashboard
- src/dashboard/** is a separate UI surface rendered in its own window with main.jsx, preload.js, and HTML/CSS assets.
- The dashboard is a React app split into sections for onboarding/wizard flow, recording status, storage, skill installation, automation, and completion.
- It communicates with the main process through IPC modules rather than direct filesystem or OS access.
4. Screen capture pipeline
- src/screen-stills/** is the heart of the product: it manages capture sessions, a queue, recorder recovery, session state, markdown extraction, and privacy handling.
- controller.js coordinates capture; recorder.js produces the stills; stills-queue.js and stills-queue-archive.js persist and recover work; session-store.js tracks session metadata.
- capture-privacy.js and on-screen-apps-detector.js enforce privacy-related decisions before capture output is stored or displayed.
5. OCR and markdown extraction
- Captured images are converted to text through src/ocr/apple-vision-ocr.js, backed by native Apple Vision OCR helpers in scripts/apple-vision-ocr.m and build scripts.
- src/screen-stills/stills-markdown-extractor.js and stills-markdown-worker.js transform OCR output into markdown, with tests covering extraction and redaction behavior.
- This is a local/offline pipeline; no cloud OCR service is part of the architecture.
6. Clipboard mirroring
- src/clipboard/** mirrors clipboard content while recording and stores it alongside still-derived context.
- Clipboard text is written into the same session-oriented context folder structure as screen markdown, so the agent sees a unified timeline of what the user saw and copied.
7. Redaction and privacy boundary
- src/security/rg-redaction.js and rg-redaction-rules.js run regex-based redaction for passwords, API keys, SSNs, credit cards, and similar secrets.
- The README notes that before markdown or clipboard text is written, Familiar runs rg-based redaction; if scanning fails twice, it still saves the file and warns once per recording session.
- build-rg-bundle.sh packages ripgrep binaries into the Electron app, so redaction is an embedded local dependency.
8. Local storage and retention
- src/storage/** manages cleanup and retention policies, including auto-session cleanup, retention-change triggers, storage usage breakdown, and window deletion behavior.
- The filesystem layout is central to the design: settings live under ~/.familiar/settings.json, and captured context is written under the user’s configured context folder, especially familiar/stills/ and familiar/stills-markdown/.
- This means the storage layer is not just persistence; it is a policy layer that keeps the local corpus bounded.
9. Settings, permissions, and onboarding
- src/settings.js, src/screen-capture/permissions.js, and the dashboard wizard rules govern first-run setup and capture permission flows.
- The dashboard contains explicit wizard sections and rule modules for permission completion, navigation, and completion state, indicating a guided setup path rather than a generic settings page.
- The app also handles low-power mode and screen-presence concerns, which are relevant to reliable background capture on macOS.
10. Agent/skill integration
- src/skills/** and src/harness-adapters/** connect Familiar’s output to external agent runtimes such as Claude Code, Codex, and Cursor.
- The built-in src/skills/familiar/SKILL.md is the default skill payload, and the adapter layer standardizes context building, prompt wrapping, and result handling for different agent tools.
- This is the outbound integration boundary: Familiar produces context that other AI tools consume.
11. OS integration and user-facing chrome
- src/tray/**, src/menu.js, src/toast/**, src/updates/**, and src/launch-intent.js provide the desktop affordances around the core capture loop.
- Tray icon state, reminders, update checks, and toast rendering are small but important subsystems because they surface capture status and operational feedback without relying on the dashboard window.
- src/startup/ownership.js suggests single-instance or ownership coordination at startup.
12. IPC as the internal contract
- src/ipc/** is the explicit contract between renderer and main process: settings, skills, stills, storage, logs, and updates are each exposed through dedicated IPC modules.
- This separation keeps the dashboard UI and the capture engine decoupled while still allowing the UI to read status, trigger actions, and display progress.
13. Tooling and test shape
- The repo is heavily test-driven, with unit tests plus Playwright E2E coverage for wizard flow, capture, storage, updates, clipboard mirroring, OCR, and permission handling.
- Build scripts and packaging assets show a macOS-focused distribution path, including native helper compilation and Electron packaging artifacts.
- The test layout mirrors the architecture closely, which makes it a good guide for drawing subsystem boundaries.
14. High-level data flow
- The main loop is: screen/clipboard capture → OCR/markdown extraction → redaction → session-based filesystem storage → IPC exposure to dashboard and agent tools.
- Around that loop are supporting flows for permissions/onboarding, retention cleanup, update notifications, and skill installation.
- For an architecture diagram, the most important boundaries are: Electron main process, React dashboard renderer, local capture/OCR/redaction pipeline, filesystem storage/context folder, and external AI agent consumers.
Main entry points: main.js Main entry · RecordingSection.jsx Dashboard sections · controller.js Capture controller · rg-redaction.js Redaction
Components 21
Electron main
Node.js main process orchestration and OS integrations
- Main entry · Bootstraps the app, windows, IPC, tray, updates, and capture lifecycle
src/main.js - Menu · Application menu and command surface
src/menu.js - Tray · Tray icon state and refresh behavior
src/tray/icon.js - Updates · Update checking and related app flow
src/updates/index.js - Settings · Loads and persists app settings
src/settings.js - IPC layer · Main-process IPC modules exposed to the dashboard
src/ipc/index.js
Dashboard renderer
React UI and preload boundary for the dashboard window
- Dashboard UI · React dashboard entry and shell composition
src/dashboard/main.jsx - Dashboard preload · Renderer bridge for safe IPC access
src/dashboard/preload.js - Dashboard sections · Wizard, recording, storage, install, automate, and completion views
src/dashboard/components/sections/RecordingSection.jsx
Capture pipeline
Screen, clipboard, OCR, markdown, and session handling
- Capture controller · Coordinates capture sessions and state
src/screen-stills/controller.js - Recorder · Produces periodic stills from the screen
src/screen-stills/recorder.js - Still queue · Queues, archives, and recovers capture work
src/screen-stills/stills-queue.js - Markdown extractor · Turns OCR output into markdown and worker jobs
src/screen-stills/stills-markdown-extractor.js - Clipboard mirror · Captures clipboard content into session context
src/clipboard/mirror.js - Apple Vision OCR · Offline OCR backed by native Apple Vision helpers
src/ocr/apple-vision-ocr.js
Storage & privacy
Local persistence, retention, and redaction controls
- Redaction · Ripgrep-based secret scanning and redaction rules
src/security/rg-redaction.js - Retention · Session cleanup, retention changes, and usage accounting
src/storage/auto-cleanup-retention.js - Context folder · Moves and structures session artifacts in the user context folder
src/context-folder/move.js
Agent integrations
Outbound skill packaging and harness adapters
- Skill installer · Installs and versions Familiar skills for downstream tools
src/skills/installer.js - Harness adapters · Wraps Familiar context for Claude Code, Codex, and Cursor
src/harness-adapters/runner.js - Familiar skill · Default skill bundle consumed by external agent runtimes
src/skills/familiar/SKILL.md
Connections 26
- Main entry → Menu: menu · Creates and updates application menu
- Main entry → Tray: tray · Manages tray icon and status
- Main entry → Updates: updates · Checks for and surfaces app updates
- Main entry → Settings: settings · Loads app configuration
- Main entry → IPC layer: IPC · Registers renderer-facing IPC handlers
- Main entry → Capture controller: capture · Starts and supervises capture pipeline
- Main entry → Clipboard mirror: clipboard · Wires clipboard capture into sessions
- Main entry → Redaction: redaction · Applies local secret scanning before persistence
- Main entry → Retention: retention · Triggers cleanup and storage policies
- Main entry → Context folder: storage · Writes and moves session artifacts
- IPC layer → Dashboard preload: bridge · Exposes safe renderer APIs
- Dashboard preload → Dashboard UI: UI IPC · Lets the dashboard call main-process actions
- Dashboard UI → Dashboard sections: sections · Composes dashboard workflow views
- Dashboard sections → IPC layer: status/actions · Reads capture, storage, skills, and update state
- Capture controller → Recorder: stills · Requests screen captures
- Capture controller → Still queue: queue · Enqueues and recovers capture work
- Capture controller → Apple Vision OCR: OCR · Converts stills to text locally
- Apple Vision OCR → Markdown extractor: text · Feeds OCR output into markdown extraction
- Markdown extractor → Redaction: sanitize · Redacts sensitive content before storage
- Clipboard mirror → Redaction: sanitize · Redacts clipboard text before writing
- Redaction → Context folder: save · Stores sanitized session artifacts
- Retention → Context folder: prune · Applies retention to local corpus
- Skill installer → Familiar skill: packages · Installs the default Familiar skill payload
- Harness adapters → Familiar skill: consumes · Wraps context for external agent runtimes
- Dashboard sections → Skill installer: install skills · Surfaces skill installation workflow
- Dashboard sections → Harness adapters: automate · Builds prompts and handoff context for agents