How oxylabs/amazon-scraper works
This repository provides a free CLI scraper that accepts an Amazon department-page URL, extracts product listings, and writes them to CSV. A separate set of Python examples demonstrates Oxylabs API requests for URL, search, product, pricing, questions, best-seller, seller, and reviews data; these examples are not wired into the CLI runtime. The runtime calls are supported by the shown source, while the API examples show requests to the external service. Other API capabilities are documented in the README but have no separate example file here.
Main entry points: collector.py Data collector · scraper.py Page scraper · __main__.py Scrape CLI · models.py Product model
Components 16
CLI runtime
- Scrape CLI
src/amazon_scraper/__main__.py - Data collector
src/amazon_scraper/collector.py
Amazon scraping
- Page scraper
src/amazon_scraper/scraper.py
Product output
- Product model
src/amazon_scraper/models.py - Products CSV
API examples
- URL request
code_examples/amazon.py - Search request
code_examples/amazon_search.py - Product request
code_examples/amazon_product.py - Pricing request
code_examples/amazon_pricing.py - Q&A request
code_examples/amazon_questions.py - Best-seller request
code_examples/amazon_bestsellers.py - Seller request
code_examples/amazon_sellers.py - Reviews request
code_examples/amazon_reviews.py
Other components
- URL provider
- Amazon pages
- Oxylabs API
Connections 15
- URL provider → Scrape CLI: provides URL
src/amazon_scraper/__main__.py - Scrape CLI → Data collector: invokes collection
src/amazon_scraper/__main__.py - Data collector → Page scraper: scrapes page
src/amazon_scraper/collector.py - Page scraper → Amazon pages: loads page
src/amazon_scraper/scraper.py - Page scraper → Product model: creates records
src/amazon_scraper/scraper.py - Data collector → Product model: serializes records
src/amazon_scraper/collector.py - Data collector → Products CSV: writes CSV
src/amazon_scraper/collector.py - URL request → Oxylabs API: posts query
code_examples/amazon.py - Search request → Oxylabs API: posts query
code_examples/amazon_search.py - Product request → Oxylabs API: posts query
code_examples/amazon_product.py - Pricing request → Oxylabs API: posts query
code_examples/amazon_pricing.py - Q&A request → Oxylabs API: posts query
code_examples/amazon_questions.py - Best-seller request → Oxylabs API: posts query
code_examples/amazon_bestsellers.py - Seller request → Oxylabs API: posts query
code_examples/amazon_sellers.py - Reviews request → Oxylabs API: posts query
code_examples/amazon_reviews.py