How artchiee/job_hunt works
Project type and purpose
• This is a standalone CLI-based web-scraping tool written in Python. Its sole responsibility is to automate job searches on the Indeed worldwide site, parse results, and persist them into a JSON file.
File structure
• Top-level files:
• Job_Scapper.py → main orchestrator script
• web_driver_configs.py → Selenium/ChromeDriver configuration
• locations.json → input lookup table for location IDs
• chromedriver → binary used by Selenium
• requirements.txt, Pipfile, Pipfile.lock → dependency manifests
• README.md, Todo.txt → documentation
• No subdirectories—this is a single-module project with clear separation between code, configuration, and data.
README for additional insights
• The README confirms it is CLI-driven, expects two optional inputs (location ID or city, and job title), and writes out a JSON file of scraped job postings.
• It uses pipenv or pip install for environment setup.
• The script internally navigates the Indeed UI via Selenium, sets the date filter to “Last 7 days,” and saves results.
a. Main components:
1. CLI Interface (User)
2. Python Runtime Environment (Virtualenv / Pipenv)
3. Job_Scapper Module (orchestrator)
4. Configuration Module (web_driver_configs.py + locations.json)
5. WebDriver (ChromeDriver binary + Selenium bindings)
6. External Service (Indeed.com)
7. Data Store (output JSON file)
b. Relationships and interactions:
• The User invokes the CLI → passes inputs into Job_Scapper.
• Job_Scapper loads configurations from web_driver_configs.py and locations.json.
• Job_Scapper instantiates Selenium WebDriver (chromedriver binary) under Python.
• WebDriver sends HTTP requests and browser commands to Indeed.com → retrieves HTML.
• Job_Scapper parses the HTML → assembles job postings objects.
• Parsed results are serialized and written to a local JSON file.
c. Architectural patterns / design principles:
• Single-responsibility: each module handles one concern (config vs scraping logic).
• Layered interaction: CLI → Orchestrator → Config → WebDriver → External site → Data store.
• Stateless between runs: no persistent database, only file I/O.
d. Key technologies and libraries:
• Python 3.x, pipenv / pip
• Selenium WebDriver, ChromeDriver
• JSON for input (locations.json) and output serialization
• requests or built-in urllib (if used) – in this case, Selenium only.