How muhammadahmadraza5088/audiocom_web_scraper works
This Scrapy project crawls Audiocom.no and emits product records, with the main spider traversing category and listing pages before extracting product details. Some spider excerpts are partial, so the graph omits uncertain links; the README is empty. The car-model CSV supports vehicle compatibility matching in the detailed scraper variants. The configured XLSX exporter provides a feed-output option; no persistent application database is shown.
Main entry points: one_page.py Single-product scraper · audiocom_1.py Category crawler · audicom_all_scraper.py All-products crawler · settings.py XLSX feed exporter
Components 12
Crawling
- Category crawler
audiocom_no/audiocom/spiders/audiocom_1.py - All-products crawler
audiocom_no/audiocom/spiders/audicom_all_scraper.py - Single-product scraper
audiocom_no/audiocom/spiders/one_page.py
Product extraction
- Product extraction
audiocom_no/audiocom/spiders/audiocom_1.py - Vehicle matching
audiocom_no/audiocom/spiders/audicom_all_scraper.py - Single-page extraction
audiocom_no/audiocom/spiders/one_page.py - Car model data
audiocom_no/car_models.csv
Record output
- XLSX feed exporter
audiocom_no/audiocom/settings.py - Product records
Other components
- Crawler operator
- Scrapy runtime
- Audiocom.no
Connections 21
- Crawler operator → Scrapy runtime: starts crawl
- Scrapy runtime → Category crawler: dispatches
- Scrapy runtime → All-products crawler: dispatches
- Scrapy runtime → Single-product scraper: dispatches
- Category crawler → Audiocom.no: requests pages
- Audiocom.no → Category crawler: returns responses
- Category crawler → Product extraction: calls parser
- Product extraction → Audiocom.no: requests products
- Audiocom.no → Product extraction: returns responses
- All-products crawler → Audiocom.no: requests pages
- Audiocom.no → All-products crawler: returns responses
- All-products crawler → Vehicle matching: calls parser
- Vehicle matching → Car model data: reads models
- Single-product scraper → Audiocom.no: requests product
- Audiocom.no → Single-product scraper: returns response
- Single-product scraper → Car model data: reads models
- Product extraction → Product records: yields items
- Vehicle matching → Product records: yields items
- Single-product scraper → Product records: yields items
- Scrapy runtime → XLSX feed exporter: configures export
- XLSX feed exporter → Product records: exports items