How vaibhavi1224/car-details-scrapper works
Main Components
• Web Scraper Engine: This is the core of the project. It consists of the Python script (notebook version in the file tree, but mentioned as a .py file in the README) that orchestrates all scraping tasks.
• Browser Automation Module: Highlight Selenium, which handles navigation to the Cars24 website, applies filters (for Tata cars & the Mumbai location), and simulates user interaction.
• HTML Parsing Module: Represent BeautifulSoup4 as the component that parses HTML content retrieved by Selenium to extract the required details (car name, kilometers driven, fuel type, transmission type, price, EMI).
• Data Storage: Show the CSV file (tata_data.csv) as the output destination where scraped data is stored.
• External Dependency: Include ChromeDriver (along with the note that it must be in the system PATH or explicitly provided) as an essential external tool for Selenium.
Relationships and Interactions
• The Web Scraper Engine initiates the process by launching the Selenium module.
• Selenium (with ChromeDriver) loads the Cars24 website and applies the necessary filters.
• The HTML content of the resultant page is passed to the BeautifulSoup parser.
• BeautifulSoup extracts the required car data and passes it back to the script.
• The script processes and saves this data into the CSV file.
Architectural Patterns & Design Principles
• Separation of Concerns: The design distinguishes between browser automation (Selenium) and data parsing (BeautifulSoup).
• Linear Batch Process: The scraper performs a one-time execution where data flows sequentially from fetching to saving.
Relevant Technologies and Frameworks
• Python 3.x is the programming language.
• Selenium for browser control.
• BeautifulSoup4 for HTML parsing.
• pandas (or Python’s CSV module if that’s used) for managing CSV file operations.
• ChromeDriver as the interface between Selenium and the Chrome browser.
Specific Elements
• A clear label for each component:
• Website (Cars24) – external system
• Selenium Module (with a note on ChromeDriver dependency)
• BeautifulSoup Module
• Data Processing Logic (possibly with a mention of pandas if that is used)
• CSV Storage (tata_data.csv)
• Selenium fetches data from the website.
• BeautifulSoup parses the HTML content from Selenium.
• Processed data flows into the CSV output.
• Annotations regarding the use of Python as the scripting language and the choice of libraries for each functionality.