How oumayma68/job-market-tracker works
Job Market Tracker collects French tech job offers, standardizes and loads them into Snowflake, checks each collection, and runs dbt transformations for business analysis. The main path is France Travail API → Airflow collection/load → raw jobs → dbt → analytics marts → Metabase. Collection also parses salary and extracts skills; dbt provides staging, curated data, and trend and salary analyses. The DAG and collector excerpts show orchestration and ingestion details. dbt model relationships and Metabase usage are represented only where the README documents them.
Main entry points: francetravail.py France Travail Collector · jobs_clean.sql Clean Jobs Mart · job_market_dag.py Airflow DAG · staging Job Staging
Components 17
Collection & Ingestion
- Airflow DAG
job_market_dag.py - France Travail Collector
collectors/francetravail.py - Salary Parsing
collectors/francetravail.py - Job Offer Schema
collectors/base.py - Skill Extraction
collectors/base.py
Warehouse & Quality
- RAW_JOBS
- Collection Quality Check
job_market_dag.py - Snowflake Warehouse
dbt Analytics
- Job Staging
dbt/models/staging - Reference Seeds
dbt/seeds - Clean Jobs Mart
dbt/models/marts/jobs_clean.sql - Job Trend Mart
dbt/models/marts/job_trend.sql - Monthly Stats Mart
dbt/models/marts/monthly_stats.sql - Skill Trends Mart
dbt/models/marts/skill_trends.sql - Skills Salary Mart
dbt/models/marts/skills_salary.sql
Business Intelligence
- Metabase Dashboard
Other components
- France Travail API
Connections 21
- Airflow DAG → France Travail Collector: invokes collection
job_market_dag.py - France Travail Collector → France Travail API: requests offers
collectors/francetravail.py - France Travail Collector → Salary Parsing: parses salary
collectors/francetravail.py - France Travail Collector → Skill Extraction: extracts skills
collectors/francetravail.py - France Travail Collector → Job Offer Schema: builds offers
collectors/francetravail.py - France Travail Collector → RAW_JOBS: loads records
collectors/francetravail.py - Airflow DAG → Collection Quality Check: checks count
job_market_dag.py - Collection Quality Check → Airflow DAG: gates dbt run
job_market_dag.py - RAW_JOBS → Job Staging: stages raw data
README.md - Job Staging → Clean Jobs Mart: curates jobs
README.md - Clean Jobs Mart → Job Trend Mart: supports analysis
- Clean Jobs Mart → Monthly Stats Mart: supports analysis
- Clean Jobs Mart → Skill Trends Mart: supports analysis
- Clean Jobs Mart → Skills Salary Mart: supports analysis
- Reference Seeds → Job Staging: provides references
- Airflow DAG → Snowflake Warehouse: configures connection
job_market_dag.py - Airflow DAG → Job Staging: runs dbt
job_market_dag.py - Snowflake Warehouse → Metabase Dashboard: serves analytics
README.md - Job Trend Mart → Metabase Dashboard: feeds visualizations
- Skill Trends Mart → Metabase Dashboard: feeds visualizations
- Skills Salary Mart → Metabase Dashboard: feeds visualizations