# dhruvgodambe/upwork-scraper architecture

> Architecture diagram of the GitHub repository dhruvgodambe/upwork-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhruvgodambe/upwork-scraper
- Repository: https://github.com/dhruvgodambe/upwork-scraper

## Overview

This is a one-shot Upwork job-discovery bot. Its main workflow authenticates in a persistent Chrome profile, builds Best Matches and keyword-search feeds, scrapes and parses job cards, stores relevant recent jobs in SQLite, then applies user-defined filters and prints matches. The graph separates orchestration, browser access, job processing, and persistence/filtering. Some feed-planning and filtering calls fall in unsampled portions of app.py; those links are supported by the README’s documented workflow.

Main entry points: [`upwork_scraper/app.py`](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py) (Run orchestration), [`upwork_scraper/filter.py`](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/filter.py) (Job filtering), [`upwork_scraper/browser.py`](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/browser.py) (Chrome startup), [`upwork_scraper/auth.py`](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/auth.py) (Upwork login)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime setup"]
  node_app["Run orchestration<br/>[app.py]"]
  node_config["Runtime settings<br/>[config.py]"]
end

subgraph group_collection["Upwork collection"]
  node_browser["Chrome startup<br/>[browser.py]"]
  node_auth["Upwork login<br/>[auth.py]"]
  node_feed["Feed planning<br/>[app.py]"]
  node_scrape["Feed scraping<br/>[app.py]"]
end

subgraph group_processing["Job processing"]
  node_parse["Job parsing<br/>[job_helpers.py]"]
  node_relevance["Relevance checks<br/>[job_helpers.py]"]
end

subgraph group_results["Persistence and results"]
  node_database[("SQLite jobs<br/>[database.py]")]
  node_filter_config["Filter rules<br/>[filter_config.json]"]
  node_filter["Job filtering<br/>[filter.py]"]
  node_display["Results display<br/>[filter.py]"]
end

node_user(("User"))
node_chrome["Chrome browser"]
node_upwork["Upwork"]

node_user -->|"runs"| node_app
node_app -->|"loads settings"| node_config
node_app -->|"launches"| node_browser
node_browser -->|"starts"| node_chrome
node_app -->|"authenticates"| node_auth
node_auth -->|"logs in"| node_upwork
node_app -->|"builds feeds"| node_feed
node_feed -->|"supplies URLs"| node_scrape
node_scrape -->|"loads job feeds"| node_upwork
node_scrape -->|"parses cards"| node_parse
node_scrape -->|"checks relevance"| node_relevance
node_app -->|"creates and writes"| node_database
node_app -->|"filters after scraping"| node_filter
node_filter_config -->|"configures"| node_filter
node_filter -->|"reads jobs"| node_database
node_filter -->|"prints matches"| node_display

click node_app "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py"
click node_config "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/config.py"
click node_browser "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/browser.py"
click node_auth "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/auth.py"
click node_feed "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py"
click node_scrape "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py"
click node_parse "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/job_helpers.py"
click node_relevance "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/job_helpers.py"
click node_database "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/database.py"
click node_filter_config "https://github.com/dhruvgodambe/upwork-scraper/blob/main/filter_config.json"
click node_filter "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/filter.py"
click node_display "https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/filter.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_config,node_user,node_chrome toneBlue
class node_browser,node_auth,node_feed,node_scrape toneAmber
class node_parse,node_relevance toneMint
class node_database,node_filter_config,node_filter,node_display toneRose
class node_upwork toneIndigo
```

## Components

### Runtime setup

- [Run orchestration](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py): `upwork_scraper/app.py`
- [Runtime settings](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/config.py): `upwork_scraper/config.py`

### Upwork collection

- [Chrome startup](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/browser.py): `upwork_scraper/browser.py`
- [Upwork login](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/auth.py): `upwork_scraper/auth.py`
- [Feed planning](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py): `upwork_scraper/app.py`
- [Feed scraping](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/app.py): `upwork_scraper/app.py`

### Job processing

- [Job parsing](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/job_helpers.py): `upwork_scraper/job_helpers.py`
- [Relevance checks](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/job_helpers.py): `upwork_scraper/job_helpers.py`

### Persistence and results

- [SQLite jobs](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/database.py): `upwork_scraper/database.py`
- [Filter rules](https://github.com/dhruvgodambe/upwork-scraper/blob/main/filter_config.json): `filter_config.json`
- [Job filtering](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/filter.py): `upwork_scraper/filter.py`
- [Results display](https://github.com/dhruvgodambe/upwork-scraper/blob/main/upwork_scraper/filter.py): `upwork_scraper/filter.py`

### Other components

- **User**
- **Chrome browser**
- **Upwork**

## Connections

- User → Run orchestration: runs (evidence: `README.md`)
- Run orchestration → Runtime settings: loads settings (evidence: `upwork_scraper/app.py`)
- Run orchestration → Chrome startup: launches (evidence: `upwork_scraper/app.py`)
- Chrome startup → Chrome browser: starts (evidence: `upwork_scraper/browser.py`)
- Run orchestration → Upwork login: authenticates (evidence: `upwork_scraper/app.py`)
- Upwork login → Upwork: logs in (evidence: `upwork_scraper/auth.py`)
- Run orchestration → Feed planning: builds feeds (evidence: `README.md`)
- Feed planning → Feed scraping: supplies URLs (evidence: `README.md`)
- Feed scraping → Upwork: loads job feeds (evidence: `README.md`)
- Feed scraping → Job parsing: parses cards (evidence: `upwork_scraper/app.py`)
- Feed scraping → Relevance checks: checks relevance (evidence: `upwork_scraper/app.py`)
- Run orchestration → SQLite jobs: creates and writes (evidence: `upwork_scraper/app.py`)
- Run orchestration → Job filtering: filters after scraping (evidence: `README.md`)
- Filter rules → Job filtering: configures (evidence: `README.md`)
- Job filtering → SQLite jobs: reads jobs (evidence: `upwork_scraper/filter.py`)
- Job filtering → Results display: prints matches (evidence: `upwork_scraper/filter.py`)
