# simonw/shot-scraper architecture

> Architecture diagram of the GitHub repository simonw/shot-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/simonw/shot-scraper
- Repository: https://github.com/simonw/shot-scraper

## Overview

shot-scraper is a command-line tool for capturing website screenshots, recording browser demos, and extracting page data with JavaScript. A user invokes the CLI with a URL or configuration; it drives a Playwright browser and produces files or data. The CLI also includes workflows for multi-shot YAML configurations and HAR resource extraction. Storyboard models validate video-demo configuration, while utility functions support URL-derived filenames and related conversions. The CLI is a large, central module; the sampled excerpts show only portions of its workflows, so unsampled feature relationships are not expanded.

Main entry points: [`shot_scraper/cli.py`](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py) (CLI commands), [`shot_scraper/video.py`](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/video.py) (Storyboard models), [`shot_scraper/utils.py`](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/utils.py) (URL utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Browser capture"]
  node_cli["CLI commands<br/>[cli.py]"]
  node_browser["Playwright browser"]
  node_web["Web pages"]
  node_screenshots["Screenshots<br/>[cli.py]"]
  node_video["Video demos<br/>[cli.py]"]
  node_files["Output files"]
end

subgraph group_automation["Automation and scraping"]
  node_multi["Multi-shot runs<br/>[cli.py]"]
  node_javascript["JavaScript scraping<br/>[cli.py]"]
  node_html["HTML output<br/>[cli.py]"]
  node_har["HAR extraction<br/>[cli.py]"]
end

subgraph group_support["Configuration and utilities"]
  node_storyboard["Storyboard models<br/>[video.py]"]
  node_utilities["URL utilities<br/>[utils.py]"]
end

node_user(("CLI user"))

node_user -->|"invokes"| node_cli
node_cli -->|"drives"| node_browser
node_browser -->|"captures"| node_web
node_cli -->|"produces"| node_screenshots
node_cli -->|"records"| node_video
node_cli -->|"loads models"| node_storyboard
node_cli -->|"runs shots"| node_multi
node_cli -->|"scrapes with"| node_javascript
node_cli -->|"extracts"| node_html
node_cli -->|"extracts entries"| node_har
node_cli -->|"uses"| node_utilities
node_cli -->|"writes"| node_files

click node_cli "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_screenshots "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_video "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_storyboard "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/video.py"
click node_multi "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_javascript "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_html "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_har "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py"
click node_utilities "https://github.com/simonw/shot-scraper/blob/main/shot_scraper/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_browser,node_web,node_screenshots,node_video,node_files,node_user toneBlue
class node_multi,node_javascript,node_html,node_har toneAmber
class node_storyboard,node_utilities toneMint
```

## Components

### Browser capture

- [CLI commands](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`
- **Playwright browser**
- **Web pages**
- [Screenshots](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`
- [Video demos](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`
- **Output files**

### Automation and scraping

- [Multi-shot runs](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`
- [JavaScript scraping](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`
- [HTML output](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`
- [HAR extraction](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/cli.py): `shot_scraper/cli.py`

### Configuration and utilities

- [Storyboard models](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/video.py): `shot_scraper/video.py`
- [URL utilities](https://github.com/simonw/shot-scraper/blob/main/shot_scraper/utils.py): `shot_scraper/utils.py`

### Other components

- **CLI user**

## Connections

- CLI user → CLI commands: invokes (evidence: `README.md`)
- CLI commands → Playwright browser: drives (evidence: `shot_scraper/cli.py`)
- Playwright browser → Web pages: captures (evidence: `README.md`)
- CLI commands → Screenshots: produces (evidence: `README.md`)
- CLI commands → Video demos: records (evidence: `README.md`)
- CLI commands → Storyboard models: loads models (evidence: `shot_scraper/cli.py`)
- CLI commands → Multi-shot runs: runs shots (evidence: `shot_scraper/cli.py`)
- CLI commands → JavaScript scraping: scrapes with (evidence: `README.md`)
- CLI commands → HTML output: extracts (evidence: `shot_scraper/cli.py`)
- CLI commands → HAR extraction: extracts entries (evidence: `shot_scraper/cli.py`)
- CLI commands → URL utilities: uses (evidence: `shot_scraper/cli.py`)
- CLI commands → Output files: writes (evidence: `README.md`)
