How zerolithx/bdjobscreapper works
BD Job Notification aggregates listings from two Bangladesh job websites, normalizes and deduplicates them into a JSON store, then exposes that data through a static frontend. The principal workflow is scheduled scraping: fetch source pages, parse and classify listings, merge them with existing records, sort and persist, and let job seekers browse and follow original application links. The frontend implementation was not sampled, so its internal interactions are limited to README-supported behavior.
Components 11
Automation
- Six-Hour Scheduler
.github/workflows/ci-and-scrape.yml
Collection
- Scrape Orchestrator
scrapper/scrape_sources.py - HTTP Fetcher
scrapper/scrape_sources.py - BD Parser
scrapper/scrape_sources.py - CV Guy Parser
scrapper/scrape_sources.py - Record Helpers
scrapper/scrape_sources.py
Storage
- Jobs JSON Store
data/jobs.json
Presentation
- Static Frontend
index.html
Other components
- Job Seeker
- BD Govt Job
- The CV Guy
Connections 14
- Six-Hour Scheduler → Scrape Orchestrator: dispatches
- Scrape Orchestrator → HTTP Fetcher: calls
- HTTP Fetcher → BD Govt Job: fetches pages
- HTTP Fetcher → The CV Guy: fetches pages
- Scrape Orchestrator → BD Parser: parses listings
- Scrape Orchestrator → CV Guy Parser: parses listings
- BD Parser → Record Helpers: normalizes records
- CV Guy Parser → Record Helpers: normalizes records
- Scrape Orchestrator → Jobs JSON Store: reads existing
- Scrape Orchestrator → Jobs JSON Store: writes records
- Static Frontend → Jobs JSON Store: reads listings
- Job Seeker → Static Frontend: browses jobs
- Static Frontend → BD Govt Job: links applications
- Static Frontend → The CV Guy: links applications