How shubbham28/job-hunt-agent works
Project Type and Purpose
• Type: A standalone Python-based job-search/data-processing pipeline.
• Purpose:
– Scrape job listings from external sites
– Parse the user’s CV to extract skills/experience
– Deduplicate and score jobs against the CV (ATS scoring)
– Suggest roles based on parsed CV
– Output ranked jobs (CSV) and optionally email results
File Structure
• Top-Level Orchestrator:
– main.py: ties all pipeline stages together
– job_hunt_agent_notebook.ipynb: exploratory data analysis and prototyping
• Configuration:
– config/credentials_config.yaml: holds API keys, email creds
• Input Artifacts:
– cv_input/your_cv.pdf
• Data Store (local JSON/CSV files):
– data/raw_jobs.json, data/filtered_jobs.json, data/jobs_history.json
– data/cv_parsed.txt, data/suggested_roles.txt
– output/ats_ranked_jobs.csv
• Processing Modules (under scripts/):
– scrape_jobs.py – fetch raw job postings
– deduplicate_jobs.py – remove duplicates
– parse_cv.py – extract skills from CV
– score_ats.py – rank jobs vs. CV (ATS scoring)
– suggest_roles.py – suggest new roles based on CV profile
– email_jobs.py – send ranked jobs via email
• Utility Code (under utils/):
– scrape_utils.py – shared HTTP/request helpers
README for Insights
• Minimal README; core details gleaned from directory names and script purposes
a. Main Components
1. External Data Sources
– Job boards / company sites (HTTP/Scraping targets)
– SMTP/email server
2. Configuration Store
– credentials_config.yaml
3. Ingestion Layer
– scrape_jobs module
4. Processing Layer (Pipeline Stages)
– deduplicate_jobs
– parse_cv
– score_ats
– suggest_roles
5. Data Storage
– JSON files under data/
– CSV output under output/
6. Notification Layer
– email_jobs module
7. Orchestrator
– main.py (orchestrates calls to each script in sequence)
8. Notebook
– job_hunt_agent_notebook.ipynb (offline/side-by-side analysis)
b. Relationships & Data Flow
1. Config → Orchestrator & modules (credentials)
2. Orchestrator → scrape_jobs → raw_jobs.json
3. raw_jobs.json → deduplicate_jobs → filtered_jobs.json
4. CV input → parse_cv → cv_parsed.txt
5. filtered_jobs.json + cv_parsed.txt → score_ats → ats_ranked_jobs.csv
6. cv_parsed.txt → suggest_roles → suggested_roles.txt
7. ats_ranked_jobs.csv → email_jobs → SMTP server → user inbox
c. Architectural Patterns & Principles
– Modular pipeline (single-responsibility scripts)
– File-based staging (JSON/CSV) for transparency & restarting
– Central configuration file for credentials
– Separation of concerns: scraping, processing, notification
d. Key Technologies
– Python 3 scripts
– YAML config
– JSON/CSV data stores
– Jupyter Notebook
– External HTTP targets (job boards)
– SMTP/email server
Detail
• Every script as its own node, even if brief
• Config injection into each module that needs credentials
• Both intermediate files and final outputs
• Frequency/triggers (e.g., “run daily via cron”)