How benjaminbeni1/book-catalogue-scraper works
This is a small, single-file catalogue scraper. Running scraper.py fetches catalogue pages with retries and delays, extracts and normalizes book fields into an in-memory collection, then writes the collection to SQLite and CSV. The README also documents pagination and skipped failed pages. The runtime is monolithic, so the diagram separates its stages by responsibility rather than implying separate modules; all code-stage paths point to the inspected scraper.py.
Main entry points: scraper.py Scraper entry point
Components 8
Scraping runtime
- Scraper entry point
scraper.py - Page fetching
scraper.py - HTML extraction
scraper.py - Book records
scraper.py
Data outputs
- SQLite database
scraper.py - CSV export
scraper.py
Other components
- Terminal user
- Book catalogue
Connections 8
- Terminal user → Scraper entry point: runs
- Scraper entry point → Page fetching: fetches pages
- Page fetching → Book catalogue: requests pages
- Book catalogue → Page fetching: returns HTML
- Scraper entry point → HTML extraction: parses HTML
- HTML extraction → Book records: extracts fields
- Scraper entry point → SQLite database: writes records
- Scraper entry point → CSV export: exports records