# muhammadahmadraza5088/audiocom_web_scraper architecture

> Architecture diagram of the GitHub repository muhammadahmadraza5088/audiocom_web_scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muhammadahmadraza5088/audiocom_web_scraper
- Repository: https://github.com/muhammadahmadraza5088/audiocom_web_scraper

## Overview

This Scrapy project crawls Audiocom.no and emits product records, with the main spider traversing category and listing pages before extracting product details. Some spider excerpts are partial, so the graph omits uncertain links; the README is empty. The car-model CSV supports vehicle compatibility matching in the detailed scraper variants. The configured XLSX exporter provides a feed-output option; no persistent application database is shown.

Main entry points: [`audiocom_no/audiocom/spiders/one_page.py`](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/one_page.py) (Single-product scraper), [`audiocom_no/audiocom/spiders/audiocom_1.py`](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audiocom_1.py) (Category crawler), [`audiocom_no/audiocom/spiders/audicom_all_scraper.py`](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audicom_all_scraper.py) (All-products crawler), [`audiocom_no/audiocom/settings.py`](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/settings.py) (XLSX feed exporter)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_crawl["Crawling"]
  node_main_spider["Category crawler<br/>[audiocom_1.py]"]
  node_all_spider["All-products crawler"]
  node_one_page["Single-product scraper<br/>[one_page.py]"]
end

subgraph group_extract["Product extraction"]
  node_product_extract["Product extraction<br/>[audiocom_1.py]"]
  node_compatibility["Vehicle matching"]
  node_single_extract["Single-page extraction<br/>[one_page.py]"]
  node_model_data["Car model data<br/>[car_models.csv]"]
end

subgraph group_output["Record output"]
  node_xlsx["XLSX feed exporter<br/>[settings.py]"]
  node_records["Product records"]
end

node_operator(("Crawler operator"))
node_runtime["Scrapy runtime"]
node_site["Audiocom.no"]

node_operator -->|"starts crawl"| node_runtime
node_runtime -->|"dispatches"| node_main_spider
node_runtime -->|"dispatches"| node_all_spider
node_runtime -->|"dispatches"| node_one_page
node_main_spider -->|"requests pages"| node_site
node_site -->|"returns responses"| node_main_spider
node_main_spider -->|"calls parser"| node_product_extract
node_product_extract -->|"requests products"| node_site
node_site -->|"returns responses"| node_product_extract
node_all_spider -->|"requests pages"| node_site
node_site -->|"returns responses"| node_all_spider
node_all_spider -->|"calls parser"| node_compatibility
node_compatibility -->|"reads models"| node_model_data
node_one_page -->|"requests product"| node_site
node_site -->|"returns response"| node_one_page
node_one_page -->|"reads models"| node_model_data
node_product_extract -->|"yields items"| node_records
node_compatibility -->|"yields items"| node_records
node_one_page -->|"yields items"| node_records
node_runtime -->|"configures export"| node_xlsx
node_xlsx -->|"exports items"| node_records

click node_main_spider "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audiocom_1.py"
click node_all_spider "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audicom_all_scraper.py"
click node_one_page "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/one_page.py"
click node_product_extract "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audiocom_1.py"
click node_compatibility "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audicom_all_scraper.py"
click node_single_extract "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/one_page.py"
click node_model_data "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/car_models.csv"
click node_xlsx "https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/settings.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main_spider,node_all_spider,node_one_page toneBlue
class node_product_extract,node_compatibility,node_single_extract,node_model_data toneAmber
class node_xlsx,node_records toneMint
class node_operator,node_runtime,node_site toneIndigo
```

## Components

### Crawling

- [Category crawler](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audiocom_1.py): `audiocom_no/audiocom/spiders/audiocom_1.py`
- [All-products crawler](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audicom_all_scraper.py): `audiocom_no/audiocom/spiders/audicom_all_scraper.py`
- [Single-product scraper](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/one_page.py): `audiocom_no/audiocom/spiders/one_page.py`

### Product extraction

- [Product extraction](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audiocom_1.py): `audiocom_no/audiocom/spiders/audiocom_1.py`
- [Vehicle matching](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/audicom_all_scraper.py): `audiocom_no/audiocom/spiders/audicom_all_scraper.py`
- [Single-page extraction](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/spiders/one_page.py): `audiocom_no/audiocom/spiders/one_page.py`
- [Car model data](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/car_models.csv): `audiocom_no/car_models.csv`

### Record output

- [XLSX feed exporter](https://github.com/muhammadahmadraza5088/audiocom_web_scraper/blob/master/audiocom_no/audiocom/settings.py): `audiocom_no/audiocom/settings.py`
- **Product records**

### Other components

- **Crawler operator**
- **Scrapy runtime**
- **Audiocom.no**

## Connections

- Crawler operator → Scrapy runtime: starts crawl
- Scrapy runtime → Category crawler: dispatches
- Scrapy runtime → All-products crawler: dispatches
- Scrapy runtime → Single-product scraper: dispatches
- Category crawler → Audiocom.no: requests pages
- Audiocom.no → Category crawler: returns responses
- Category crawler → Product extraction: calls parser
- Product extraction → Audiocom.no: requests products
- Audiocom.no → Product extraction: returns responses
- All-products crawler → Audiocom.no: requests pages
- Audiocom.no → All-products crawler: returns responses
- All-products crawler → Vehicle matching: calls parser
- Vehicle matching → Car model data: reads models
- Single-product scraper → Audiocom.no: requests product
- Audiocom.no → Single-product scraper: returns response
- Single-product scraper → Car model data: reads models
- Product extraction → Product records: yields items
- Vehicle matching → Product records: yields items
- Single-product scraper → Product records: yields items
- Scrapy runtime → XLSX feed exporter: configures export
- XLSX feed exporter → Product records: exports items