How andre3oo0/football-analytics works
This repository is a football analytics data pipeline: scheduled or manual runs fetch competition data, land it in DuckDB, transform it through dbt into a tested dimensional model, and optionally export marts as Parquet for Power BI. The graph follows documented pipeline stages and sampled ingestion/export code; individual dbt model dependencies and workflow commands are based on README documentation, since those implementation files were not sampled.
Main entry points: intermediate Intermediate views · pipeline.yml Pipeline workflow · run.py Ingestion runner · api_client.py API client
Components 18
Orchestration
- Pipeline workflow
.github/workflows/pipeline.yml
Ingestion
- Ingestion runner
ingestion/run.py - API client
ingestion/api_client.py - Raw JSON cache
data/raw - Raw loader
ingestion/loader.py
Warehouse Modeling
- DuckDB warehouse
- Staging views
dbt/models/staging - Intermediate views
dbt/models/intermediate - Competition seed
dbt/seeds/seed_competitions.csv - Dimension tables
dbt/models/marts - Match facts
dbt/models/marts/fact_matches.sql - Derived standings
dbt/models/marts/fact_standings.sql - dbt data tests
dbt/tests
Serving
- Mart exporter
export_marts.py - Parquet exports
exports
Other components
- Schedule / manual trigger
- Football-data.org API
- Power BI consumer
Connections 20
- Schedule / manual trigger → Pipeline workflow: triggers
- Pipeline workflow → Ingestion runner: runs ingestion
- Pipeline workflow → Staging views: builds dbt
- Ingestion runner → API client: requests data
- API client → Football-data.org API: fetches
- API client → Raw JSON cache: reads / writes
- Ingestion runner → Raw loader: loads responses
- Raw loader → DuckDB warehouse: upserts raw
- DuckDB warehouse → Staging views: reads raw
- Staging views → Intermediate views: transforms
- Intermediate views → Dimension tables: builds dimensions
- Competition seed → Dimension tables: supplies reference
- Intermediate views → Match facts: builds match fact
- Intermediate views → Derived standings: derives standings
- Dimension tables → dbt data tests: tested by
- Match facts → dbt data tests: tested by
- Derived standings → dbt data tests: tested by
- DuckDB warehouse → Mart exporter: reads marts
- Mart exporter → Parquet exports: writes files
- Parquet exports → Power BI consumer: consumed by