How rashmi0405/ai-model-tracker works
AI Model Price Tracker is a daily ELT pipeline that captures OpenRouter model data, preserves raw snapshots, transforms them into price history and analytics tables, and serves those marts to a Looker Studio dashboard. Airflow orders extraction, loading, freshness checks, and dbt builds. The tree contains no snapshot configuration or dashboard implementation, so those capabilities are represented from the README with no source path; transformation relationships use documented architecture rather than unsampled SQL details.
Main entry points: ai_model_tracker_dag.py Daily Airflow DAG · stg_openrouter__models.sql Typed staging view · int_openrouter__latest_models.sql Latest models view · extract_models.py Snapshot extraction
Components 15
Scheduling
- Daily Airflow DAG
ai_model_tracker_dag.py
Data ingestion
- Snapshot extraction
extract_models.py - Raw Cloud Storage
- BigQuery loader
load_to_bigquery.py - BigQuery raw table
dbt transformations
- Source freshness check
sources.yml - Typed staging view
stg_openrouter__models.sql - Latest models view
int_openrouter__latest_models.sql - SCD Type 2 history
- Model dimension
dim_models.sql - Daily price fact
fct_model_daily_prices.sql - Price change fact
fct_model_price_changes.sql - Provider daily aggregate
agg_provider_daily_summary.sql
Analytics serving
- Looker Studio dashboard
Other components
- OpenRouter API
Connections 20
- Daily Airflow DAG → Snapshot extraction: runs task
ai_model_tracker_dag.py - Daily Airflow DAG → BigQuery loader: runs task
ai_model_tracker_dag.py - Daily Airflow DAG → Source freshness check: runs check
ai_model_tracker_dag.py - Daily Airflow DAG → Typed staging view: runs dbt build
ai_model_tracker_dag.py - OpenRouter API → Snapshot extraction: provides models
extract_models.py - Snapshot extraction → Raw Cloud Storage: writes snapshot
extract_models.py - Raw Cloud Storage → BigQuery loader: supplies snapshot
load_to_bigquery.py - BigQuery loader → BigQuery raw table: loads rows
load_to_bigquery.py - BigQuery raw table → Source freshness check: checks freshness
README.md - BigQuery raw table → Typed staging view: transforms raw data
README.md - Typed staging view → Latest models view: selects latest day
README.md - Typed staging view → SCD Type 2 history: tracks price versions
README.md - Latest models view → Model dimension: builds marts
README.md - SCD Type 2 history → Daily price fact: builds daily fact
README.md - SCD Type 2 history → Price change fact: derives changes
README.md - Latest models view → Provider daily aggregate: builds aggregate
README.md - Model dimension → Looker Studio dashboard: serves mart tables
README.md - Daily price fact → Looker Studio dashboard: serves mart tables
README.md - Price change fact → Looker Studio dashboard: serves mart tables
README.md - Provider daily aggregate → Looker Studio dashboard: serves mart tables
README.md