# oumayma68/job-market-tracker architecture

> Architecture diagram of the GitHub repository oumayma68/job-market-tracker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oumayma68/job-market-tracker
- Repository: https://github.com/oumayma68/job-market-tracker

## Overview

Job Market Tracker collects French tech job offers, standardizes and loads them into Snowflake, checks each collection, and runs dbt transformations for business analysis. The main path is France Travail API → Airflow collection/load → raw jobs → dbt → analytics marts → Metabase. Collection also parses salary and extracts skills; dbt provides staging, curated data, and trend and salary analyses. The DAG and collector excerpts show orchestration and ingestion details. dbt model relationships and Metabase usage are represented only where the README documents them.

Main entry points: [`collectors/francetravail.py`](https://github.com/oumayma68/job-market-tracker/blob/main/collectors/francetravail.py) (France Travail Collector), [`dbt/models/marts/jobs_clean.sql`](https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/jobs_clean.sql) (Clean Jobs Mart), [`job_market_dag.py`](https://github.com/oumayma68/job-market-tracker/blob/main/job_market_dag.py) (Airflow DAG), [`dbt/models/staging`](https://github.com/oumayma68/job-market-tracker/tree/main/dbt/models/staging) (Job Staging)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Collection &amp; Ingestion"]
  node_airflow_dag["Airflow DAG<br/>[job_market_dag.py]"]
  node_collector["France Travail Collector<br/>[francetravail.py]"]
  node_salary_parser["Salary Parsing<br/>[francetravail.py]"]
  node_job_schema["Job Offer Schema<br/>[base.py]"]
  node_skill_extraction["Skill Extraction<br/>[base.py]"]
end

subgraph group_warehouse["Warehouse &amp; Quality"]
  node_raw_jobs[("RAW_JOBS")]
  node_quality_check["Collection Quality Check<br/>[job_market_dag.py]"]
  node_snowflake[("Snowflake Warehouse")]
end

subgraph group_analytics["dbt Analytics"]
  node_dbt_staging["Job Staging"]
  node_reference_seeds["Reference Seeds"]
  node_jobs_clean["Clean Jobs Mart<br/>[jobs_clean.sql]"]
  node_job_trend["Job Trend Mart<br/>[job_trend.sql]"]
  node_monthly_stats["Monthly Stats Mart<br/>[monthly_stats.sql]"]
  node_skill_trends["Skill Trends Mart<br/>[skill_trends.sql]"]
  node_skills_salary["Skills Salary Mart<br/>[skills_salary.sql]"]
end

subgraph group_bi["Business Intelligence"]
  node_metabase["Metabase Dashboard"]
end

node_france_api(("France Travail API"))

node_airflow_dag -->|"invokes collection"| node_collector
node_collector -->|"requests offers"| node_france_api
node_collector -->|"parses salary"| node_salary_parser
node_collector -->|"extracts skills"| node_skill_extraction
node_collector -->|"builds offers"| node_job_schema
node_collector -->|"loads records"| node_raw_jobs
node_airflow_dag -->|"checks count"| node_quality_check
node_quality_check -->|"gates dbt run"| node_airflow_dag
node_raw_jobs -->|"stages raw data"| node_dbt_staging
node_dbt_staging -->|"curates jobs"| node_jobs_clean
node_jobs_clean -.->|"supports analysis"| node_job_trend
node_jobs_clean -.->|"supports analysis"| node_monthly_stats
node_jobs_clean -.->|"supports analysis"| node_skill_trends
node_jobs_clean -.->|"supports analysis"| node_skills_salary
node_reference_seeds -.->|"provides references"| node_dbt_staging
node_airflow_dag -->|"configures connection"| node_snowflake
node_airflow_dag -->|"runs dbt"| node_dbt_staging
node_snowflake -->|"serves analytics"| node_metabase
node_job_trend -.->|"feeds visualizations"| node_metabase
node_skill_trends -.->|"feeds visualizations"| node_metabase
node_skills_salary -.->|"feeds visualizations"| node_metabase

click node_airflow_dag "https://github.com/oumayma68/job-market-tracker/blob/main/job_market_dag.py"
click node_collector "https://github.com/oumayma68/job-market-tracker/blob/main/collectors/francetravail.py"
click node_salary_parser "https://github.com/oumayma68/job-market-tracker/blob/main/collectors/francetravail.py"
click node_job_schema "https://github.com/oumayma68/job-market-tracker/blob/main/collectors/base.py"
click node_skill_extraction "https://github.com/oumayma68/job-market-tracker/blob/main/collectors/base.py"
click node_quality_check "https://github.com/oumayma68/job-market-tracker/blob/main/job_market_dag.py"
click node_dbt_staging "https://github.com/oumayma68/job-market-tracker/tree/main/dbt/models/staging"
click node_reference_seeds "https://github.com/oumayma68/job-market-tracker/tree/main/dbt/seeds"
click node_jobs_clean "https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/jobs_clean.sql"
click node_job_trend "https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/job_trend.sql"
click node_monthly_stats "https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/monthly_stats.sql"
click node_skill_trends "https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/skill_trends.sql"
click node_skills_salary "https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/skills_salary.sql"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_airflow_dag,node_collector,node_salary_parser,node_job_schema,node_skill_extraction toneBlue
class node_raw_jobs,node_quality_check,node_snowflake toneAmber
class node_dbt_staging,node_reference_seeds,node_jobs_clean,node_job_trend,node_monthly_stats,node_skill_trends,node_skills_salary,node_france_api toneMint
class node_metabase toneRose
```

## Components

### Collection & Ingestion

- [Airflow DAG](https://github.com/oumayma68/job-market-tracker/blob/main/job_market_dag.py): `job_market_dag.py`
- [France Travail Collector](https://github.com/oumayma68/job-market-tracker/blob/main/collectors/francetravail.py): `collectors/francetravail.py`
- [Salary Parsing](https://github.com/oumayma68/job-market-tracker/blob/main/collectors/francetravail.py): `collectors/francetravail.py`
- [Job Offer Schema](https://github.com/oumayma68/job-market-tracker/blob/main/collectors/base.py): `collectors/base.py`
- [Skill Extraction](https://github.com/oumayma68/job-market-tracker/blob/main/collectors/base.py): `collectors/base.py`

### Warehouse & Quality

- **RAW_JOBS**
- [Collection Quality Check](https://github.com/oumayma68/job-market-tracker/blob/main/job_market_dag.py): `job_market_dag.py`
- **Snowflake Warehouse**

### dbt Analytics

- [Job Staging](https://github.com/oumayma68/job-market-tracker/tree/main/dbt/models/staging): `dbt/models/staging`
- [Reference Seeds](https://github.com/oumayma68/job-market-tracker/tree/main/dbt/seeds): `dbt/seeds`
- [Clean Jobs Mart](https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/jobs_clean.sql): `dbt/models/marts/jobs_clean.sql`
- [Job Trend Mart](https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/job_trend.sql): `dbt/models/marts/job_trend.sql`
- [Monthly Stats Mart](https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/monthly_stats.sql): `dbt/models/marts/monthly_stats.sql`
- [Skill Trends Mart](https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/skill_trends.sql): `dbt/models/marts/skill_trends.sql`
- [Skills Salary Mart](https://github.com/oumayma68/job-market-tracker/blob/main/dbt/models/marts/skills_salary.sql): `dbt/models/marts/skills_salary.sql`

### Business Intelligence

- **Metabase Dashboard**

### Other components

- **France Travail API**

## Connections

- Airflow DAG → France Travail Collector: invokes collection (evidence: `job_market_dag.py`)
- France Travail Collector → France Travail API: requests offers (evidence: `collectors/francetravail.py`)
- France Travail Collector → Salary Parsing: parses salary (evidence: `collectors/francetravail.py`)
- France Travail Collector → Skill Extraction: extracts skills (evidence: `collectors/francetravail.py`)
- France Travail Collector → Job Offer Schema: builds offers (evidence: `collectors/francetravail.py`)
- France Travail Collector → RAW_JOBS: loads records (evidence: `collectors/francetravail.py`)
- Airflow DAG → Collection Quality Check: checks count (evidence: `job_market_dag.py`)
- Collection Quality Check → Airflow DAG: gates dbt run (evidence: `job_market_dag.py`)
- RAW_JOBS → Job Staging: stages raw data (evidence: `README.md`)
- Job Staging → Clean Jobs Mart: curates jobs (evidence: `README.md`)
- Clean Jobs Mart → Job Trend Mart: supports analysis
- Clean Jobs Mart → Monthly Stats Mart: supports analysis
- Clean Jobs Mart → Skill Trends Mart: supports analysis
- Clean Jobs Mart → Skills Salary Mart: supports analysis
- Reference Seeds → Job Staging: provides references
- Airflow DAG → Snowflake Warehouse: configures connection (evidence: `job_market_dag.py`)
- Airflow DAG → Job Staging: runs dbt (evidence: `job_market_dag.py`)
- Snowflake Warehouse → Metabase Dashboard: serves analytics (evidence: `README.md`)
- Job Trend Mart → Metabase Dashboard: feeds visualizations
- Skill Trends Mart → Metabase Dashboard: feeds visualizations
- Skills Salary Mart → Metabase Dashboard: feeds visualizations
