# senaye2003/soccer-spark-analysis architecture

> Architecture diagram of the GitHub repository senaye2003/soccer-spark-analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/senaye2003/soccer-spark-analysis
- Repository: https://github.com/senaye2003/soccer-spark-analysis

## Overview

This repository is a PySpark soccer-event analysis project. Its main workflow loads StatsBomb match/event data, shapes it into Spark datasets, persists events and match metadata, then runs analytical queries; separate entry points provide shot-to-goal ML and a simulated Structured Streaming demonstration. The graph reflects the README’s documented analytical scope, while the excerpts show implementation details primarily for ingestion, queries, ML, and streaming; documented but unimplemented capabilities are shown without speculative call edges.

Main entry points: [`src/transformations.py`](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py) (Query Runner), [`src/ml_pipeline.py`](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py) (ML Runner), [`src/streaming.py`](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py) (Streaming Runner), [`src/ingestion.py`](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py) (Ingestion Job)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion"]
  node_ingest_job["Ingestion Job<br/>[ingestion.py]"]
  node_event_data[("Event Dataset<br/>[ingestion.py]")]
  node_match_data[("Match Dimension<br/>[ingestion.py]")]
  node_eda["Ingestion EDA<br/>[ingestion.py]"]
end

subgraph group_analysis["Batch Analysis"]
  node_query_job["Query Runner<br/>[transformations.py]"]
  node_shot_efficiency["Shot Efficiency<br/>[transformations.py]"]
  node_team_comparison["Team Comparison<br/>[transformations.py]"]
  node_passing["Passing Analysis<br/>[transformations.py]"]
  node_event_trends["Event Trends<br/>[transformations.py]"]
  node_match_goals["Match Goal Join<br/>[transformations.py]"]
  node_passing_network["Passing Networks"]
  node_pressure_analysis["Pressure Analysis"]
end

subgraph group_ml["Shot Modeling"]
  node_ml_job["ML Runner<br/>[ml_pipeline.py]"]
  node_shot_features["Shot Features<br/>[ml_pipeline.py]"]
  node_goal_model["Goal Classifier<br/>[ml_pipeline.py]"]
  node_ml_metrics["Model Metrics<br/>[ml_pipeline.py]"]
  node_role_clustering["Player Role Clustering"]
end

subgraph group_stream["Streaming Demo"]
  node_stream_job["Streaming Runner<br/>[streaming.py]"]
  node_stream_input["Simulated Feed<br/>[streaming.py]"]
  node_team_aggregation["Team Aggregation<br/>[streaming.py]"]
  node_console["Console Output<br/>[streaming.py]"]
end

node_analyst(("Analyst"))
node_statsbomb["StatsBomb Open Data"]

node_analyst -->|"runs"| node_ingest_job
node_statsbomb -->|"provides data"| node_ingest_job
node_ingest_job -->|"runs"| node_eda
node_ingest_job -->|"writes"| node_event_data
node_ingest_job -->|"writes"| node_match_data
node_analyst -->|"runs"| node_query_job
node_query_job -->|"reads"| node_event_data
node_query_job -->|"reads"| node_match_data
node_query_job -->|"runs"| node_shot_efficiency
node_query_job -->|"runs"| node_team_comparison
node_query_job -->|"runs"| node_passing
node_query_job -->|"runs"| node_event_trends
node_query_job -->|"runs"| node_match_goals
node_analyst -.->|"runs"| node_ml_job
node_ml_job -->|"reads"| node_event_data
node_ml_job -->|"derives"| node_shot_features
node_ml_job -->|"trains"| node_goal_model
node_ml_job -->|"reports"| node_ml_metrics
node_analyst -.->|"runs"| node_stream_job
node_stream_job -->|"reads"| node_event_data
node_stream_job -->|"writes"| node_stream_input
node_stream_job -->|"aggregates"| node_team_aggregation
node_stream_job -->|"writes"| node_console
node_analyst -.->|"seeks"| node_passing_network
node_analyst -.->|"seeks"| node_pressure_analysis
node_analyst -.->|"seeks"| node_role_clustering

click node_ingest_job "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py"
click node_event_data "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py"
click node_match_data "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py"
click node_eda "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py"
click node_query_job "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py"
click node_shot_efficiency "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py"
click node_team_comparison "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py"
click node_passing "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py"
click node_event_trends "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py"
click node_match_goals "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py"
click node_ml_job "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py"
click node_shot_features "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py"
click node_goal_model "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py"
click node_ml_metrics "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py"
click node_stream_job "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py"
click node_stream_input "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py"
click node_team_aggregation "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py"
click node_console "https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py"
click node_passing_network "https://github.com/senaye2003/soccer-spark-analysis/tree/main/src"
click node_pressure_analysis "https://github.com/senaye2003/soccer-spark-analysis/tree/main/src"
click node_role_clustering "https://github.com/senaye2003/soccer-spark-analysis/tree/main/src"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_job,node_event_data,node_match_data,node_eda toneBlue
class node_query_job,node_shot_efficiency,node_team_comparison,node_passing,node_event_trends,node_match_goals,node_passing_network,node_pressure_analysis toneAmber
class node_ml_job,node_shot_features,node_goal_model,node_ml_metrics,node_role_clustering toneMint
class node_stream_job,node_stream_input,node_team_aggregation,node_console toneRose
class node_analyst,node_statsbomb toneIndigo
```

## Components

### Ingestion

- [Ingestion Job](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py): `src/ingestion.py`
- [Event Dataset](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py): `src/ingestion.py`
- [Match Dimension](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py): `src/ingestion.py`
- [Ingestion EDA](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ingestion.py): `src/ingestion.py`

### Batch Analysis

- [Query Runner](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py): `src/transformations.py`
- [Shot Efficiency](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py): `src/transformations.py`
- [Team Comparison](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py): `src/transformations.py`
- [Passing Analysis](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py): `src/transformations.py`
- [Event Trends](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py): `src/transformations.py`
- [Match Goal Join](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/transformations.py): `src/transformations.py`
- [Passing Networks](https://github.com/senaye2003/soccer-spark-analysis/tree/main/src): `src`
- [Pressure Analysis](https://github.com/senaye2003/soccer-spark-analysis/tree/main/src): `src`

### Shot Modeling

- [ML Runner](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py): `src/ml_pipeline.py`
- [Shot Features](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py): `src/ml_pipeline.py`
- [Goal Classifier](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py): `src/ml_pipeline.py`
- [Model Metrics](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/ml_pipeline.py): `src/ml_pipeline.py`
- [Player Role Clustering](https://github.com/senaye2003/soccer-spark-analysis/tree/main/src): `src`

### Streaming Demo

- [Streaming Runner](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py): `src/streaming.py`
- [Simulated Feed](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py): `src/streaming.py`
- [Team Aggregation](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py): `src/streaming.py`
- [Console Output](https://github.com/senaye2003/soccer-spark-analysis/blob/main/src/streaming.py): `src/streaming.py`

### Other components

- **Analyst**
- **StatsBomb Open Data**

## Connections

- Analyst → Ingestion Job: runs (evidence: `README.md`)
- StatsBomb Open Data → Ingestion Job: provides data (evidence: `src/ingestion.py`)
- Ingestion Job → Ingestion EDA: runs (evidence: `src/ingestion.py`)
- Ingestion Job → Event Dataset: writes (evidence: `src/ingestion.py`)
- Ingestion Job → Match Dimension: writes (evidence: `src/ingestion.py`)
- Analyst → Query Runner: runs (evidence: `README.md`)
- Query Runner → Event Dataset: reads (evidence: `src/transformations.py`)
- Query Runner → Match Dimension: reads (evidence: `src/transformations.py`)
- Query Runner → Shot Efficiency: runs (evidence: `src/transformations.py`)
- Query Runner → Team Comparison: runs (evidence: `src/transformations.py`)
- Query Runner → Passing Analysis: runs (evidence: `src/transformations.py`)
- Query Runner → Event Trends: runs (evidence: `src/transformations.py`)
- Query Runner → Match Goal Join: runs (evidence: `src/transformations.py`)
- Analyst → ML Runner: runs
- ML Runner → Event Dataset: reads (evidence: `src/ml_pipeline.py`)
- ML Runner → Shot Features: derives (evidence: `src/ml_pipeline.py`)
- ML Runner → Goal Classifier: trains (evidence: `src/ml_pipeline.py`)
- ML Runner → Model Metrics: reports (evidence: `src/ml_pipeline.py`)
- Analyst → Streaming Runner: runs
- Streaming Runner → Event Dataset: reads (evidence: `src/streaming.py`)
- Streaming Runner → Simulated Feed: writes (evidence: `src/streaming.py`)
- Streaming Runner → Team Aggregation: aggregates (evidence: `src/streaming.py`)
- Streaming Runner → Console Output: writes (evidence: `src/streaming.py`)
- Analyst → Passing Networks: seeks
- Analyst → Pressure Analysis: seeks
- Analyst → Player Role Clustering: seeks