# sakho115/f1_azure architecture

> Architecture diagram of the GitHub repository sakho115/f1_azure, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sakho115/f1_azure
- Repository: https://github.com/sakho115/f1_azure

## Overview

This is a local-first Formula 1 analytics lakehouse that ingests historical and API-sourced data, processes it through Bronze, Silver, and Gold tiers, validates the resulting datasets, and presents analytics in a Streamlit dashboard. The documented main workflow is driven by the Python orchestrator; Silver transformations and merges produce the inputs to Gold marts. The graph also shows dashboard views and governance as product capabilities. Some connections are documented rather than directly visible in sampled code, and the API integration is not shown wired into the main ingestion path.

Main entry points: [`orchestration/pipeline.py`](https://github.com/sakho115/f1_azure/blob/main/orchestration/pipeline.py) (Pipeline Orchestrator), [`dashboard/app.py`](https://github.com/sakho115/f1_azure/blob/main/dashboard/app.py) (Streamlit Dashboard), [`data`](https://github.com/sakho115/f1_azure/tree/main/data) (Silver Tables), [`src/silver/merge_manager.py`](https://github.com/sakho115/f1_azure/blob/main/src/silver/merge_manager.py) (Silver Merge Manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Pipeline Orchestration"]
  node_orchestrator["Pipeline Orchestrator<br/>[pipeline.py]"]
  node_spark["Spark Session<br/>[spark.py]"]
end

subgraph group_lakehouse["Lakehouse Data"]
  node_samples["Historical Inputs"]
  node_ingestion["Bronze Ingestion<br/>[file_ingestion.py]"]
  node_bronze[("Bronze Tables")]
  node_transforms["Silver Transformations"]
  node_merge["Silver Merge Manager<br/>[merge_manager.py]"]
  node_silver[("Silver Tables")]
end

subgraph group_analytics["Analytics and Quality"]
  node_standings["Standings Marts<br/>[standings.py]"]
  node_telemetry["Telemetry Marts<br/>[telemetry.py]"]
  node_gold[("Gold Tables")]
  node_quality["Data Quality<br/>[expectations.py]"]
  node_governance["SQL Governance<br/>[governance.sql]"]
end

subgraph group_serving["Dashboard Serving"]
  node_dashboard["Streamlit Dashboard<br/>[app.py]"]
  node_standings_view["Standings View<br/>[standings_view.py]"]
  node_telemetry_view["Telemetry View<br/>[telemetry_view.py]"]
  node_circuit_view["Circuit View<br/>[circuit_view.py]"]
end

node_operator(("Pipeline Operator"))
node_api(("Ergast API"))

node_operator -->|"starts"| node_orchestrator
node_orchestrator -->|"initializes"| node_spark
node_orchestrator -->|"runs Bronze"| node_ingestion
node_ingestion -->|"reads fallback"| node_samples
node_ingestion -->|"writes Delta"| node_bronze
node_api -.->|"may supply data"| node_ingestion
node_orchestrator -->|"runs Silver"| node_merge
node_merge -->|"reads Delta"| node_bronze
node_merge -->|"applies transforms"| node_transforms
node_merge -->|"merges tables"| node_silver
node_orchestrator -->|"builds standings"| node_standings
node_orchestrator -->|"builds telemetry"| node_telemetry
node_standings -->|"reads tables"| node_silver
node_telemetry -->|"reads tables"| node_silver
node_standings -->|"writes marts"| node_gold
node_telemetry -->|"writes marts"| node_gold
node_orchestrator -->|"runs validation"| node_quality
node_quality -->|"checks tables"| node_silver
node_quality -->|"checks tables"| node_gold
node_dashboard -->|"loads analytics"| node_gold
node_dashboard -->|"loads circuits"| node_silver
node_dashboard -->|"renders"| node_standings_view
node_dashboard -->|"renders"| node_telemetry_view
node_dashboard -->|"renders"| node_circuit_view

click node_samples "https://github.com/sakho115/f1_azure/tree/main/data/sample"
click node_orchestrator "https://github.com/sakho115/f1_azure/blob/main/orchestration/pipeline.py"
click node_spark "https://github.com/sakho115/f1_azure/blob/main/src/utils/spark.py"
click node_ingestion "https://github.com/sakho115/f1_azure/blob/main/src/ingestion/file_ingestion.py"
click node_bronze "https://github.com/sakho115/f1_azure/tree/main/data"
click node_transforms "https://github.com/sakho115/f1_azure/tree/main/src/transformations"
click node_merge "https://github.com/sakho115/f1_azure/blob/main/src/silver/merge_manager.py"
click node_silver "https://github.com/sakho115/f1_azure/tree/main/data"
click node_standings "https://github.com/sakho115/f1_azure/blob/main/src/gold/standings.py"
click node_telemetry "https://github.com/sakho115/f1_azure/blob/main/src/gold/telemetry.py"
click node_gold "https://github.com/sakho115/f1_azure/tree/main/data"
click node_quality "https://github.com/sakho115/f1_azure/blob/main/src/quality/expectations.py"
click node_dashboard "https://github.com/sakho115/f1_azure/blob/main/dashboard/app.py"
click node_standings_view "https://github.com/sakho115/f1_azure/blob/main/dashboard/components/standings_view.py"
click node_telemetry_view "https://github.com/sakho115/f1_azure/blob/main/dashboard/components/telemetry_view.py"
click node_circuit_view "https://github.com/sakho115/f1_azure/blob/main/dashboard/components/circuit_view.py"
click node_governance "https://github.com/sakho115/f1_azure/blob/main/sql/governance.sql"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_orchestrator,node_spark toneBlue
class node_samples,node_ingestion,node_bronze,node_transforms,node_merge,node_silver toneAmber
class node_standings,node_telemetry,node_gold,node_quality,node_governance,node_api toneMint
class node_dashboard,node_standings_view,node_telemetry_view,node_circuit_view toneRose
class node_operator toneIndigo
```

## Components

### Pipeline Orchestration

- [Pipeline Orchestrator](https://github.com/sakho115/f1_azure/blob/main/orchestration/pipeline.py): `orchestration/pipeline.py`
- [Spark Session](https://github.com/sakho115/f1_azure/blob/main/src/utils/spark.py): `src/utils/spark.py`

### Lakehouse Data

- [Historical Inputs](https://github.com/sakho115/f1_azure/tree/main/data/sample): `data/sample`
- [Bronze Ingestion](https://github.com/sakho115/f1_azure/blob/main/src/ingestion/file_ingestion.py): `src/ingestion/file_ingestion.py`
- [Bronze Tables](https://github.com/sakho115/f1_azure/tree/main/data): `data`
- [Silver Transformations](https://github.com/sakho115/f1_azure/tree/main/src/transformations): `src/transformations`
- [Silver Merge Manager](https://github.com/sakho115/f1_azure/blob/main/src/silver/merge_manager.py): `src/silver/merge_manager.py`
- [Silver Tables](https://github.com/sakho115/f1_azure/tree/main/data): `data`

### Analytics and Quality

- [Standings Marts](https://github.com/sakho115/f1_azure/blob/main/src/gold/standings.py): `src/gold/standings.py`
- [Telemetry Marts](https://github.com/sakho115/f1_azure/blob/main/src/gold/telemetry.py): `src/gold/telemetry.py`
- [Gold Tables](https://github.com/sakho115/f1_azure/tree/main/data): `data`
- [Data Quality](https://github.com/sakho115/f1_azure/blob/main/src/quality/expectations.py): `src/quality/expectations.py`
- [SQL Governance](https://github.com/sakho115/f1_azure/blob/main/sql/governance.sql): `sql/governance.sql`

### Dashboard Serving

- [Streamlit Dashboard](https://github.com/sakho115/f1_azure/blob/main/dashboard/app.py): `dashboard/app.py`
- [Standings View](https://github.com/sakho115/f1_azure/blob/main/dashboard/components/standings_view.py): `dashboard/components/standings_view.py`
- [Telemetry View](https://github.com/sakho115/f1_azure/blob/main/dashboard/components/telemetry_view.py): `dashboard/components/telemetry_view.py`
- [Circuit View](https://github.com/sakho115/f1_azure/blob/main/dashboard/components/circuit_view.py): `dashboard/components/circuit_view.py`

### Other components

- **Pipeline Operator**
- **Ergast API**

## Connections

- Pipeline Operator → Pipeline Orchestrator: starts (evidence: `README.md`)
- Pipeline Orchestrator → Spark Session: initializes (evidence: `orchestration/pipeline.py`)
- Pipeline Orchestrator → Bronze Ingestion: runs Bronze (evidence: `orchestration/pipeline.py`)
- Bronze Ingestion → Historical Inputs: reads fallback (evidence: `src/ingestion/file_ingestion.py`)
- Bronze Ingestion → Bronze Tables: writes Delta (evidence: `src/ingestion/file_ingestion.py`)
- Ergast API → Bronze Ingestion: may supply data
- Pipeline Orchestrator → Silver Merge Manager: runs Silver (evidence: `orchestration/pipeline.py`)
- Silver Merge Manager → Bronze Tables: reads Delta (evidence: `src/silver/merge_manager.py`)
- Silver Merge Manager → Silver Transformations: applies transforms (evidence: `src/silver/merge_manager.py`)
- Silver Merge Manager → Silver Tables: merges tables (evidence: `src/silver/merge_manager.py`)
- Pipeline Orchestrator → Standings Marts: builds standings (evidence: `orchestration/pipeline.py`)
- Pipeline Orchestrator → Telemetry Marts: builds telemetry (evidence: `orchestration/pipeline.py`)
- Standings Marts → Silver Tables: reads tables (evidence: `src/gold/standings.py`)
- Telemetry Marts → Silver Tables: reads tables (evidence: `src/gold/telemetry.py`)
- Standings Marts → Gold Tables: writes marts (evidence: `src/gold/standings.py`)
- Telemetry Marts → Gold Tables: writes marts (evidence: `src/gold/telemetry.py`)
- Pipeline Orchestrator → Data Quality: runs validation (evidence: `orchestration/pipeline.py`)
- Data Quality → Silver Tables: checks tables (evidence: `src/quality/expectations.py`)
- Data Quality → Gold Tables: checks tables (evidence: `src/quality/expectations.py`)
- Streamlit Dashboard → Gold Tables: loads analytics (evidence: `dashboard/app.py`)
- Streamlit Dashboard → Silver Tables: loads circuits (evidence: `dashboard/app.py`)
- Streamlit Dashboard → Standings View: renders (evidence: `dashboard/app.py`)
- Streamlit Dashboard → Telemetry View: renders (evidence: `dashboard/app.py`)
- Streamlit Dashboard → Circuit View: renders (evidence: `dashboard/app.py`)
