# vaibhavi1224/car-details-scrapper architecture

> Architecture diagram of the GitHub repository vaibhavi1224/car-details-scrapper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vaibhavi1224/car-details-scrapper
- Repository: https://github.com/vaibhavi1224/car-details-scrapper

## Overview

##### Main Components
- Web Scraper Engine: This is the core of the project. It consists of the Python script (notebook version in the file tree, but mentioned as a .py file in the README) that orchestrates all scraping tasks.
- Browser Automation Module: Highlight Selenium, which handles navigation to the Cars24 website, applies filters (for Tata cars & the Mumbai location), and simulates user interaction.
- HTML Parsing Module: Represent BeautifulSoup4 as the component that parses HTML content retrieved by Selenium to extract the required details (car name, kilometers driven, fuel type, transmission type, price, EMI).
- Data Storage: Show the CSV file (tata_data.csv) as the output destination where scraped data is stored.
- External Dependency: Include ChromeDriver (along with the note that it must be in the system PATH or explicitly provided) as an essential external tool for Selenium.

##### Relationships and Interactions
- The Web Scraper Engine initiates the process by launching the Selenium module.
- Selenium (with ChromeDriver) loads the Cars24 website and applies the necessary filters.
- The HTML content of the resultant page is passed to the BeautifulSoup parser.
- BeautifulSoup extracts the required car data and passes it back to the script.
- The script processes and saves this data into the CSV file.

##### Architectural Patterns & Design Principles
- Separation of Concerns: The design distinguishes between browser automation (Selenium) and data parsing (BeautifulSoup).
- Linear Batch Process: The scraper performs a one-time execution where data flows sequentially from fetching to saving.

##### Relevant Technologies and Frameworks
- Python 3.x is the programming language.
- Selenium for browser control.
- BeautifulSoup4 for HTML parsing.
- pandas (or Python’s CSV module if that’s used) for managing CSV file operations.
- ChromeDriver as the interface between Selenium and the Chrome browser.

##### Specific Elements
- A clear label for each component:
- Website (Cars24) – external system
- Selenium Module (with a note on ChromeDriver dependency)
- BeautifulSoup Module
- Data Processing Logic (possibly with a mention of pandas if that is used)
- CSV Storage (tata_data.csv)
- Selenium fetches data from the website.
- BeautifulSoup parses the HTML content from Selenium.
- Processed data flows into the CSV output.
- Annotations regarding the use of Python as the scripting language and the choice of libraries for each functionality.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External System
    E["Cars24 Website (Mumbai)"]:::external

    %% Scraper Orchestration Layer
    subgraph "Scraper Orchestration"
        A["Web Scraper Engine (Python3)"]:::engine
    end

    %% Browser Automation and HTML Parsing Layer
    subgraph "Browser & Parsing"
        B["Selenium Module (with ChromeDriver)"]:::automation
        C["BeautifulSoup Module"]:::parsing
    end

    %% Data Storage Layer
    subgraph "Data Storage"
        D["CSV File (tata_data.csv)"]:::data
    end

    %% Data Flow
    A -->|"launches"| B
    B -->|"fetches_HTML_from"| E
    B -->|"passes_HTML_to"| C
    C -->|"returns_parsed_data_to"| A
    A -->|"writes_data_to"| D

    %% Click Events
    click A "https://github.com/vaibhavi1224/car-details-scrapper/blob/main/scrape_tata_mumbai_cars24.ipynb"
    click B "https://github.com/vaibhavi1224/car-details-scrapper/blob/main/scrape_tata_mumbai_cars24.ipynb"
    click C "https://github.com/vaibhavi1224/car-details-scrapper/blob/main/scrape_tata_mumbai_cars24.ipynb"
    click D "https://github.com/vaibhavi1224/car-details-scrapper/blob/main/tata_data.csv"

    %% Styles
    classDef engine fill:#f9c,stroke:#333,stroke-width:2px;
    classDef automation fill:#bbf,stroke:#333,stroke-width:2px;
    classDef parsing fill:#bfb,stroke:#333,stroke-width:2px;
    classDef data fill:#ff9,stroke:#333,stroke-width:2px;
    classDef external fill:#faa,stroke:#333,stroke-width:2px;
```
