# smudgelord200/cb_web_scraper architecture

> Architecture diagram of the GitHub repository smudgelord200/cb_web_scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smudgelord200/cb_web_scraper
- Repository: https://github.com/smudgelord200/cb_web_scraper

## Overview

This is a scheduled-style event discovery script: running main.py initializes spaCy, gathers event listings from configured sites, filters for active Cate Blanchett involvement, compares matching URLs with notification history, saves a current-results snapshot, and emails newly found matches. The graph preserves static and dynamic fetching, parsing/relevance analysis, persistence, and reporting. README describes the broader capability set; source excerpts do not expose every scraper branch, so only evidenced call relationships are drawn.

Main entry points: [`main.py`](https://github.com/smudgelord200/cb_web_scraper/blob/master/main.py) (Run coordinator), [`data_storage.py`](https://github.com/smudgelord200/cb_web_scraper/blob/master/data_storage.py) (JSON storage), [`scraper.py`](https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py) (Site scraper), [`email_notification.py`](https://github.com/smudgelord200/cb_web_scraper/blob/master/email_notification.py) (Email delivery)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Run orchestration"]
  node_main["Run coordinator<br/>[main.py]"]
  node_email["Email delivery"]
end

subgraph group_discovery["Event discovery"]
  node_scrape["Site scraper<br/>[scraper.py]"]
  node_staticfetch["Static fetch<br/>[scraper.py]"]
  node_eventparse["Event extraction<br/>[scraper.py]"]
  node_relevance["Involvement analysis<br/>[scraper.py]"]
  node_browserfetch["Browser fetching"]
end

subgraph group_state["State and reporting"]
  node_store["JSON storage<br/>[data_storage.py]"]
  node_snapshot[("Current event log")]
  node_history[("Notified URL history")]
  node_recipients["Recipient list"]
  node_console["Event console output<br/>[print_message.py]"]
end

node_operator(("Operator"))
node_mailservice(("Email service"))
node_eventwebsites(("Event websites"))

node_operator -->|"runs"| node_main
node_main -->|"requests scan"| node_scrape
node_scrape -->|"fetches HTML"| node_staticfetch
node_scrape -->|"requests browser HTML"| node_browserfetch
node_staticfetch -->|"fetches pages"| node_eventwebsites
node_browserfetch -->|"loads pages"| node_eventwebsites
node_scrape -->|"parses listings"| node_eventparse
node_scrape -->|"checks involvement"| node_relevance
node_main -->|"loads history"| node_store
node_store -->|"reads URLs"| node_history
node_main -->|"saves snapshot"| node_store
node_store -->|"writes events"| node_snapshot
node_main -->|"loads recipients"| node_store
node_store -->|"reads list"| node_recipients
node_main -->|"updates history"| node_store
node_main -->|"prints events"| node_console
node_main -->|"sends new matches"| node_email
node_email -->|"delivers email"| node_mailservice

click node_main "https://github.com/smudgelord200/cb_web_scraper/blob/master/main.py"
click node_scrape "https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py"
click node_staticfetch "https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py"
click node_eventparse "https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py"
click node_relevance "https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py"
click node_browserfetch "https://github.com/smudgelord200/cb_web_scraper/blob/master/selenium_scraper_methods.py"
click node_store "https://github.com/smudgelord200/cb_web_scraper/blob/master/data_storage.py"
click node_console "https://github.com/smudgelord200/cb_web_scraper/blob/master/print_message.py"
click node_email "https://github.com/smudgelord200/cb_web_scraper/blob/master/email_notification.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_email toneBlue
class node_scrape,node_staticfetch,node_eventparse,node_relevance,node_browserfetch toneAmber
class node_store,node_snapshot,node_history,node_recipients,node_console toneMint
class node_operator,node_mailservice,node_eventwebsites toneIndigo
```

## Components

### Run orchestration

- [Run coordinator](https://github.com/smudgelord200/cb_web_scraper/blob/master/main.py): `main.py`
- [Email delivery](https://github.com/smudgelord200/cb_web_scraper/blob/master/email_notification.py): `email_notification.py`

### Event discovery

- [Site scraper](https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py): `scraper.py`
- [Static fetch](https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py): `scraper.py`
- [Event extraction](https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py): `scraper.py`
- [Involvement analysis](https://github.com/smudgelord200/cb_web_scraper/blob/master/scraper.py): `scraper.py`
- [Browser fetching](https://github.com/smudgelord200/cb_web_scraper/blob/master/selenium_scraper_methods.py): `selenium_scraper_methods.py`

### State and reporting

- [JSON storage](https://github.com/smudgelord200/cb_web_scraper/blob/master/data_storage.py): `data_storage.py`
- **Current event log**
- **Notified URL history**
- **Recipient list**
- [Event console output](https://github.com/smudgelord200/cb_web_scraper/blob/master/print_message.py): `print_message.py`

### Other components

- **Operator**
- **Email service**
- **Event websites**

## Connections

- Operator → Run coordinator: runs (evidence: `README.md`)
- Run coordinator → Site scraper: requests scan (evidence: `main.py`)
- Site scraper → Static fetch: fetches HTML (evidence: `scraper.py`)
- Site scraper → Browser fetching: requests browser HTML (evidence: `scraper.py`)
- Static fetch → Event websites: fetches pages (evidence: `scraper.py`)
- Browser fetching → Event websites: loads pages (evidence: `selenium_scraper_methods.py`)
- Site scraper → Event extraction: parses listings (evidence: `scraper.py`)
- Site scraper → Involvement analysis: checks involvement (evidence: `scraper.py`)
- Run coordinator → JSON storage: loads history (evidence: `main.py`)
- JSON storage → Notified URL history: reads URLs (evidence: `data_storage.py`)
- Run coordinator → JSON storage: saves snapshot (evidence: `main.py`)
- JSON storage → Current event log: writes events (evidence: `data_storage.py`)
- Run coordinator → JSON storage: loads recipients (evidence: `main.py`)
- JSON storage → Recipient list: reads list (evidence: `data_storage.py`)
- Run coordinator → JSON storage: updates history (evidence: `main.py`)
- Run coordinator → Event console output: prints events (evidence: `main.py`)
- Run coordinator → Email delivery: sends new matches (evidence: `main.py`)
- Email delivery → Email service: delivers email (evidence: `email_notification.py`)
