# sawlachintan/cricket_data architecture

> Architecture diagram of the GitHub repository sawlachintan/cricket_data, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sawlachintan/cricket_data
- Repository: https://github.com/sawlachintan/cricket_data

## Overview

##### Project’s Purpose and Type
- The project focuses on the ingestion, processing, and analysis of cricket data from various leagues/series.
- It follows a multi-stage pipeline that categorizes data into Bronze, Silver, and Gold engineering layers, and finally into an analysis layer where visualizations (dashboards) and prediction models run.
- The purpose is twofold: to transform raw cricket data (stored in CSVs and Parquet files under data/bronze and data/util directories) into refined datasets, and to enable further analysis for insights and predictions.

##### File Structure
- Key directories are:
- data/bronze: Contains raw data segregated by cricket leagues/series (such as bbl, bpl, cpl, etc.), with each league having multiple files (event_df, info_df, innings_df, etc.).
- data/util: Contains utility CSV files summarizing or supporting information across leagues.
- The presence of Python scripts like fetch_data.py and json_to_csv.py indicates parts of the pipeline for data extraction and transformation.
- The existence of requirements.txt suggests dependency management.
- There is no explicit frontend or backend directory, which reinforces that this is a data engineering pipeline project primarily.

##### Utilize the README Insights
- A high-level engineering process showing the flow from Bronze to Silver to Gold, and eventually connecting to the Analysis components (visualizations and prediction).

##### Main Components
a. Data Ingestion:
- Web Layer/Endpoint represented by fetch_data.py.
- Process that converts JSON to CSV (json_to_csv.py), tying into the data processing pipeline.
b. Data Processing Pipeline (Engineering Layer):
- Bronze: Raw data ingestion from various leagues (as structured under data/bronze).
- Silver and Gold: Intermediate and final stages of data processing/aggregation, even if the physical directories are not present, they are implied by the pipeline flow.
c. Data Analysis:
- Visualization (dashboards) module.
- Prediction or modeling module.

##### Relationships and Interactions Between Components
- The Web layer (fetch_data) initiates data collection and then passes data to json_to_csv, which converts data from JSON into CSV.
- Converted data feeds into the Bronze stage where raw datasets are organized by different cricket leagues.
- Processed data then move organically to the Silver stage and eventually to the Gold stage where refined “basic_stats” and other metrics are computed.
- Finally, the refined data is used by the Analysis layer for visualizations and predictive analytics.

##### Architectural Patterns and Design Principles
- Pipeline pattern: Data moves in a sequential manner through distinct stages (Bronze → Silver → Gold → Analysis).
- Separation of concerns: Ingestion, transformation, and analysis are clearly separated.
- Layered Architecture: Different layers of processing; engineering stages and analysis functions are visually and conceptually divided.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        FD("Fetch Data (Web Layer)"):::ingestion
        JC("JSON to CSV Converter"):::ingestion
    end

    %% Engineering Pipeline
    subgraph "Engineering Pipeline"
        BR("Bronze Data Storage (Raw Cricket Data)"):::storage
        SI("Silver Data Processing (Intermediate)"):::processing
        GO("Gold Data Aggregation (Refined Data)"):::processing
    end

    %% Data Analysis Layer
    subgraph "Data Analysis Layer"
        VD("Visualization Dashboard"):::analysis
        PR("Prediction Engine"):::analysis
    end

    UT("Utility Data (Supporting Info)"):::storage

    %% Data Flow Connections
    FD -- "triggers" --> JC
    JC -- "feedsData" --> BR
    UT -- "supports" --> BR
    BR -- "dataFlow" --> SI
    SI -- "dataFlow" --> GO
    GO -- "drives" --> VD
    GO -- "drives" --> PR

    %% Click Events
    click FD "https://github.com/sawlachintan/cricket_data/blob/main/fetch_data.py"
    click JC "https://github.com/sawlachintan/cricket_data/blob/main/json_to_csv.py"
    click BR "https://github.com/sawlachintan/cricket_data/tree/main/data/bronze"
    click UT "https://github.com/sawlachintan/cricket_data/tree/main/data/util"

    %% Styles
    classDef ingestion fill:#ccf,stroke:#333,stroke-width:2px;
    classDef storage fill:#ffc,stroke:#333,stroke-width:2px;
    classDef processing fill:#cfc,stroke:#333,stroke-width:2px;
    classDef analysis fill:#fcf,stroke:#333,stroke-width:2px;
```
