How tranletrungkien1705/us-ai-jobs-crawler works
This repository runs two job-discovery workflows: a US-focused crawler that tags AI-automatable roles and upserts them to Supabase, and a Vietnam .NET job collector that enriches and accumulates listings for job boards. The README documents the US sources and daily Actions schedule; sampled code shows additional sources and the VN collection stages. The board pages and some crawler details are represented from the file tree or documentation, not inspected source excerpts.
Main entry points: crawl.py US crawler · vn_crawl.py VN crawler · vn_scrape.js Browser scraper · vn.html Vietnam job board
Components 18
US AI Jobs
- US crawler
crawl.py - US job feeds
crawl.py - Role tagging
crawl.py
Vietnam .NET Jobs
- VN crawler
vn_crawl.py - LinkedIn jobs
- ITviec jobs
- Remote feeds
- Fit enrichment
vn_crawl.py - Browser scraper
vn_scrape.js - TopCV jobs
- VietnamWorks jobs
Persistence and Boards
- Supabase jobs
schema.sql - VN job archive
docs/vn-jobs.json - Scraped job archive
docs/topcv-jobs.json - Vietnam job board
docs/vn.html - US jobs board
docs/index.html
Other components
- Job seeker
- Daily Actions
.github/workflows/crawl-jobs.yml
Connections 17
- Daily Actions → US crawler: runs daily
README.md - US crawler → US job feeds: fetches listings
crawl.py - US job feeds → US crawler: returns jobs
crawl.py - US crawler → Role tagging: tags and filters
crawl.py - US crawler → Supabase jobs: upserts jobs
crawl.py - US crawler → US jobs board: publishes results
README.md - Job seeker → Vietnam job board: views listings
- VN crawler → LinkedIn jobs: fetches jobs
vn_crawl.py - VN crawler → ITviec jobs: fetches jobs
vn_crawl.py - VN crawler → Remote feeds: fetches jobs
vn_crawl.py - VN crawler → Fit enrichment: classifies listings
vn_crawl.py - VN crawler → VN job archive: merges and writes
vn_crawl.py - Browser scraper → TopCV jobs: scrapes listings
vn_scrape.js - Browser scraper → VietnamWorks jobs: scrapes listings
vn_scrape.js - Browser scraper → Scraped job archive: merges and writes
vn_scrape.js - VN job archive → Vietnam job board: supplies listings
- Scraped job archive → Vietnam job board: supplies listings