How wajih1234/job-scraper works
This is a small Tunisian job-listing scraper with a Streamlit dashboard. The main user workflow searches Tanitjobs for a job title, collects and parses matching offers, saves them to SQLite, then queries the database for title matches and filters by city. A separate batch workflow crawls links, scrapes offers into JSON, and imports them into the same database; a command-line script searches that database by title or description. The README documents the dashboard and local database; the batch workflow is evident in the sampled scripts.
Main entry points: app.py Streamlit Dashboard · scraper.py Batch Parser · tanit.py Offer Writer · get_links.py Page Crawler
Components 11
Job Discovery
- Keyword Search
tanit.py - Page Crawler
get_links.py
Offer Ingestion
- Offer Parser
tanit.py - Batch Parser
scraper.py
Local Storage
- Offer Writer
tanit.py - JSON Importer
to_db.py - Jobs Database
Search Interfaces
- Streamlit Dashboard
app.py - CLI Search
search.py
Other components
- Job Seeker
- Tanitjobs
Connections 15
- Job Seeker → Streamlit Dashboard: uses
README.md - Streamlit Dashboard → Keyword Search: requests offers
app.py - Keyword Search → Tanitjobs: fetches listings
tanit.py - Streamlit Dashboard → Offer Writer: saves links
app.py - Offer Writer → Offer Parser: parses offers
tanit.py - Offer Parser → Tanitjobs: fetches details
tanit.py - Offer Writer → Jobs Database: writes offers
tanit.py - Streamlit Dashboard → Jobs Database: queries and filters
app.py - Job Seeker → CLI Search: searches offers
search.py - CLI Search → Jobs Database: queries offers
search.py - Page Crawler → Tanitjobs: crawls listings
get_links.py - Page Crawler → Batch Parser: writes links for
get_links.py - Batch Parser → Tanitjobs: fetches details
scraper.py - Batch Parser → JSON Importer: writes offers for
scraper.py - JSON Importer → Jobs Database: imports offers
to_db.py