How r3ner3n/nodeclipper-for-notebooklm works
1. Overall architecture
- This is a Chrome Extension (Manifest V3), not a standalone web app.
- It is fully client-side: there is no backend server or external API in the core architecture.
- The extension is built from TypeScript with Vite and packaged into a dist/ folder for loading into Chrome.
2. Primary runtime pieces
- Side Panel UI: the user-facing control surface where crawling starts, selection happens, and export is triggered.
- Background Service Worker: the orchestration layer that performs crawling, manages the discovered site tree, and handles export/download actions.
- Offscreen Document: a hidden DOM-capable context used for HTML parsing and conversion, because the service worker cannot do DOM-based extraction directly.
3. UI boundary: src/side_panel
- panel.ts and panel.css implement the interactive panel shown on the right side of Chrome.
- This UI owns the main user workflow: set crawl depth, start crawling, inspect the tree, select nodes, preview URLs, and export.
- The panel also includes a language toggle (JA/EN), with preference stored locally in localStorage.
4. Orchestration boundary: src/background
- background/index.ts acts as the central coordinator.
- It runs the crawl queue, applies the breadth-first crawl strategy, enforces the 500 ms request delay, and restricts traversal to the same domain.
- It also aggregates crawl results into the tree structure consumed by the side panel and prepares the export output.
5. Parsing boundary: src/offscreen
- offscreen/offscreen.ts is the document-processing layer.
- It uses Mozilla Readability to isolate article/main content from full HTML.
- It uses Turndown to convert cleaned HTML into Markdown, including link-style handling for media when appropriate.
- This separation is important because parsing depends on DOM APIs that are unavailable in the service worker.
6. Data flow: crawl to tree
- The user starts from the current page in Chrome.
- The side panel sends a crawl request to the background worker.
- The background worker fetches pages, discovers links, and builds a collapsible tree of pages.
- That tree is streamed back to the side panel for display and selection.
7. Data flow: selection to export
- The user selects individual pages or subtrees in the panel.
- Parent selection propagates to descendants, and partially selected branches show an indeterminate state.
- On export, the background worker merges the selected pages into a single Markdown file and writes outputs into a timestamped folder in Downloads.
8. Media handling
- The tree can include non-HTML nodes such as PDFs, images, audio, and video.
- These can be downloaded individually from the panel.
- The export flow keeps media separate from the merged Markdown so the final NotebookLM import can include both the .md source and any relevant files.
9. Extension surfaces and packaging
- public/manifest.json defines the MV3 extension surfaces and permissions.
- public/side_panel/index.html and public/offscreen/index.html are the HTML entrypoints for the two non-background contexts.
- public/icons/ contains the extension assets shown in Chrome.
10. Key architectural constraints
- The system is designed around browser session access, which is why it can crawl login-protected pages that external scrapers typically cannot.
- The use of a side panel + service worker + offscreen document is the core architectural pattern.
- All important state is local to the extension/runtime; there is no remote persistence layer.
11. Repository-level structure
- src/types holds shared type definitions used across the three runtime contexts.
- vite.config.ts, tsconfig.json, and package.json define the build pipeline and TypeScript packaging.
- .devcontainer/ is developer tooling only; it does not affect runtime architecture.
12. What to show in an architecture diagram
- Draw Chrome as the host boundary.
- Inside it, show three boxes: Side Panel UI, Background Service Worker, and Offscreen Document.
- Show the data path: current page → crawl queue/tree in background → tree state in side panel → selection/export → downloads folder.
- Show the parsing path: fetched HTML → offscreen Readability/Turndown → Markdown.
Main entry points: index.ts Background SW · panel.ts Side Panel · offscreen.ts Offscreen Doc · manifest.json Manifest
Components 17
Chrome host
Extension runtime surfaces inside Chrome
- Chrome · Browser host boundary for the extension
- Side Panel · User control surface for crawling, selection, preview, and export
src/side_panel/panel.ts - Background SW · Central coordinator for crawl, tree building, and export
src/background/index.ts - Offscreen Doc · DOM-capable parsing and Markdown conversion context
src/offscreen/offscreen.ts - Current Page · Starting point for session-aware crawling
- Crawl Queue · Breadth-first crawl worklist with delay and same-domain rules
- Site Tree · Collapsible discovered page tree shared with the panel
- Selection · Chosen pages and subtrees with indeterminate propagation
- Export Job · Builds the final Markdown bundle from selected pages
- Markdown File · Merged Markdown produced for NotebookLM import
- Downloads · Chrome download destination for export and media files
- Media Files · Non-HTML downloads such as PDFs, images, audio, and video
- Readability · Isolates main article content from fetched HTML
- Turndown · Converts cleaned HTML into Markdown with link handling
Build & packaging
Vite/TypeScript build and extension assets
- Manifest · Defines extension surfaces, permissions, and entrypoints
public/manifest.json - Vite Build · TypeScript/Vite packaging into the dist folder
vite.config.ts
Other components
- Shared Types · Common interfaces used across all runtime contexts
src/types/index.ts
Connections 27
- Chrome → Side Panel: hosts · Chrome hosts the side panel surface
- Chrome → Background SW: hosts · Chrome runs the service worker
- Chrome → Offscreen Doc: hosts · Chrome provides the hidden document context
- Side Panel → Background SW: crawl request · User starts crawling from the panel
- Current Page → Background SW: seed URL · Crawl begins from the active tab/session page
- Background SW → Crawl Queue: enqueue · Background worker schedules breadth-first crawl work
- Crawl Queue → Background SW: dequeue · Queued pages are fetched with domain and delay rules
- Background SW → Site Tree: build tree · Discovered pages are aggregated into the site tree
- Site Tree → Side Panel: stream state · Tree state is rendered for inspection and selection
- Side Panel → Selection: select nodes · Parent and descendant selection is managed in the UI
- Selection → Background SW: export request · Selected pages are handed to the export pipeline
- Background SW → Offscreen Doc: parse HTML · Fetched page HTML is sent to the offscreen document
- Offscreen Doc → Readability: extract content · Readability isolates the main article or content region
- Readability → Turndown: clean HTML · Cleaned content is prepared for Markdown conversion
- Turndown → Markdown File: render markdown · Converted Markdown is assembled for export
- Background SW → Markdown File: merge files · Background worker merges selected pages into one Markdown file
- Markdown File → Downloads: write export · Timestamped export folder is written to Downloads
- Background SW → Media Files: download media · Media assets are saved separately from the Markdown bundle
- Media Files → Downloads: store files · Downloaded media is placed alongside the export output
- Side Panel → Media Files: single download · Panel can trigger individual media downloads
- Manifest → Side Panel: declares · Manifest wires the side panel entrypoint
- Manifest → Background SW: declares · Manifest wires the background service worker
- Manifest → Offscreen Doc: declares · Manifest wires the offscreen document entrypoint
- Shared Types → Side Panel: shared contract · Shared interfaces shape panel state and messages
- Shared Types → Background SW: shared contract · Shared interfaces shape orchestration messages and tree state
- Shared Types → Offscreen Doc: shared contract · Shared interfaces shape parsing messages and outputs
- Vite Build → Manifest: packages · Build pipeline emits the extension bundle and assets