# shubbham28/job-hunt-agent architecture

> Architecture diagram of the GitHub repository shubbham28/job-hunt-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shubbham28/job-hunt-agent
- Repository: https://github.com/shubbham28/job-hunt-agent

## Overview

##### Project Type and Purpose
- Type: A standalone Python-based job-search/data-processing pipeline.
- Purpose:
– Scrape job listings from external sites
– Parse the user’s CV to extract skills/experience
– Deduplicate and score jobs against the CV (ATS scoring)
– Suggest roles based on parsed CV
– Output ranked jobs (CSV) and optionally email results

##### File Structure
- Top-Level Orchestrator:
– main.py: ties all pipeline stages together
– job_hunt_agent_notebook.ipynb: exploratory data analysis and prototyping
- Configuration:
– config/credentials_config.yaml: holds API keys, email creds
- Input Artifacts:
– cv_input/your_cv.pdf
- Data Store (local JSON/CSV files):
– data/raw_jobs.json, data/filtered_jobs.json, data/jobs_history.json
– data/cv_parsed.txt, data/suggested_roles.txt
– output/ats_ranked_jobs.csv
- Processing Modules (under scripts/):
– scrape_jobs.py – fetch raw job postings
– deduplicate_jobs.py – remove duplicates
– parse_cv.py – extract skills from CV
– score_ats.py – rank jobs vs. CV (ATS scoring)
– suggest_roles.py – suggest new roles based on CV profile
– email_jobs.py – send ranked jobs via email
- Utility Code (under utils/):
– scrape_utils.py – shared HTTP/request helpers

##### README for Insights
- Minimal README; core details gleaned from directory names and script purposes

a. Main Components
1. External Data Sources
– Job boards / company sites (HTTP/Scraping targets)
– SMTP/email server
2. Configuration Store
– credentials_config.yaml
3. Ingestion Layer
– scrape_jobs module
4. Processing Layer (Pipeline Stages)
– deduplicate_jobs
– parse_cv
– score_ats
– suggest_roles
5. Data Storage
– JSON files under data/
– CSV output under output/
6. Notification Layer
– email_jobs module
7. Orchestrator
– main.py (orchestrates calls to each script in sequence)
8. Notebook
– job_hunt_agent_notebook.ipynb (offline/side-by-side analysis)
b. Relationships & Data Flow
1. Config → Orchestrator & modules (credentials)
2. Orchestrator → scrape_jobs → raw_jobs.json
3. raw_jobs.json → deduplicate_jobs → filtered_jobs.json
4. CV input → parse_cv → cv_parsed.txt
5. filtered_jobs.json + cv_parsed.txt → score_ats → ats_ranked_jobs.csv
6. cv_parsed.txt → suggest_roles → suggested_roles.txt
7. ats_ranked_jobs.csv → email_jobs → SMTP server → user inbox
c. Architectural Patterns & Principles
– Modular pipeline (single-responsibility scripts)
– File-based staging (JSON/CSV) for transparency & restarting
– Central configuration file for credentials
– Separation of concerns: scraping, processing, notification
d. Key Technologies
– Python 3 scripts
– YAML config
– JSON/CSV data stores
– Jupyter Notebook
– External HTTP targets (job boards)
– SMTP/email server

##### Detail
- Every script as its own node, even if brief
- Config injection into each module that needs credentials
- Both intermediate files and final outputs
- Frequency/triggers (e.g., “run daily via cron”)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Configuration"
        cfg["credentials_config.yaml"]:::config
    end

    subgraph "Orchestrator"
        orchestrator["main.py"]:::orchestrator
    end

    subgraph "Scripts"
        initScripts["__init__.py"]:::module
        scrape["scrape_jobs.py"]:::module
        dedup["deduplicate_jobs.py"]:::module
        parsecv["parse_cv.py"]:::module
        score["score_ats.py"]:::module
        suggest["suggest_roles.py"]:::module
        email["email_jobs.py"]:::module
    end

    subgraph "Utilities"
        utilInit["__init__.py"]:::module
        scrapeUtil["scrape_utils.py"]:::module
    end

    subgraph "Data Stores"
        raw["raw_jobs.json"]:::data
        filtered["filtered_jobs.json"]:::data
        history["jobs_history.json"]:::data
        cvParsed["cv_parsed.txt"]:::data
        suggested["suggested_roles.txt"]:::data
        outputCSV["ats_ranked_jobs.csv"]:::data
    end

    subgraph "External Services"
        JobBoards["Job Boards / Websites"]:::external
        SMTP["SMTP Server"]:::external
        Inbox["User Inbox"]:::external
    end

    subgraph "Notebook"
        notebook["job_hunt_agent_notebook.ipynb"]:::notebook
    end

    %% Config flows
    cfg -->|provides creds| orchestrator
    cfg -->|provides creds| scrape
    cfg -->|provides creds| email

    %% Orchestration
    orchestrator -->|"run scrape_jobs"| scrape
    orchestrator -->|"run deduplicate_jobs"| dedup
    orchestrator -->|"run parse_cv"| parsecv
    orchestrator -->|"run score_ats"| score
    orchestrator -->|"run suggest_roles"| suggest
    orchestrator -->|"run email_jobs"| email

    %% Ingestion
    JobBoards -->|scrape targets| scrape
    scrape -->|"raw_jobs.json"| raw

    %% Deduplication
    raw --> dedup
    dedup -->|"filtered_jobs.json"| filtered

    %% CV Parsing
    cvInput["your_cv.pdf"]:::data
    cvInput --> parsecv
    parsecv -->|"cv_parsed.txt"| cvParsed

    %% ATS Scoring
    filtered --> score
    cvParsed --> score
    score -->|"ats_ranked_jobs.csv"| outputCSV

    %% Role Suggestion
    cvParsed --> suggest
    suggest -->|"suggested_roles.txt"| suggested

    %% Notification
    outputCSV --> email
    email -->|send email| SMTP
    SMTP -->|deliver| Inbox

    %% Utilities
    scrapeUtil --> scrape
    initScripts --> scrape
    initScripts --> dedup
    initScripts --> parsecv
    initScripts --> score
    initScripts --> suggest
    initScripts --> email

    utilInit --> scrapeUtil

    %% Notebook Access
    raw --> notebook
    filtered --> notebook
    history --> notebook
    cvParsed --> notebook
    suggested --> notebook
    outputCSV --> notebook

    %% Click Events
    click orchestrator "https://github.com/shubbham28/job-hunt-agent/blob/main/main.py"
    click notebook "https://github.com/shubbham28/job-hunt-agent/blob/main/job_hunt_agent_notebook.ipynb"
    click cfg "https://github.com/shubbham28/job-hunt-agent/blob/main/config/credentials_config.yaml"
    click cvInput "https://github.com/shubbham28/job-hunt-agent/blob/main/cv_input/your_cv.pdf"
    click raw "https://github.com/shubbham28/job-hunt-agent/blob/main/data/raw_jobs.json"
    click filtered "https://github.com/shubbham28/job-hunt-agent/blob/main/data/filtered_jobs.json"
    click history "https://github.com/shubbham28/job-hunt-agent/blob/main/data/jobs_history.json"
    click cvParsed "https://github.com/shubbham28/job-hunt-agent/blob/main/data/cv_parsed.txt"
    click suggested "https://github.com/shubbham28/job-hunt-agent/blob/main/data/suggested_roles.txt"
    click outputCSV "https://github.com/shubbham28/job-hunt-agent/blob/main/output/ats_ranked_jobs.csv"
    click scrape "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/scrape_jobs.py"
    click dedup "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/deduplicate_jobs.py"
    click parsecv "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/parse_cv.py"
    click score "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/score_ats.py"
    click suggest "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/suggest_roles.py"
    click email "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/email_jobs.py"
    click scrapeUtil "https://github.com/shubbham28/job-hunt-agent/blob/main/utils/scrape_utils.py"
    click initScripts "https://github.com/shubbham28/job-hunt-agent/blob/main/scripts/__init__.py"
    click utilInit "https://github.com/shubbham28/job-hunt-agent/blob/main/utils/__init__.py"

    %% Styles
    classDef config fill:#e8f5e9,stroke:#2e7d32
    classDef orchestrator fill:#fff9c4,stroke:#f9a825,stroke-width:2px
    classDef module fill:#bbdefb,stroke:#1565c0
    classDef data fill:#c8e6c9,stroke:#2e7d32,stroke-dasharray: 5 5
    classDef external fill:#ffcc80,stroke:#ef6c00,stroke-width:2px
    classDef notebook fill:#e1bee7,stroke:#6a1b9a,stroke-width:2px
```
