# ravikumarchavva/cricket-prediction architecture

> Architecture diagram of the GitHub repository ravikumarchavva/cricket-prediction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ravikumarchavva/cricket-prediction
- Repository: https://github.com/ravikumarchavva/cricket-prediction

## Overview

##### Overall Structure & Main Components
- Data Sources & Ingestion:
- External data sources like Cricsheet and ESPN cricket stats (as indicated by modules in “include/a_data_sources”).
- Role of Apache Airflow (the “dags” folder with t20.py and the .astro folder for additional Airflow configuration) in triggering the pipeline.
- Data Processing Pipeline:
- Sequential stages:
- Data Preprocessing: Represent the cleaning and transformation stage (data_manuplation/1_preprocessing and tasks in “include/b_data_preprocessing”).
- Exploratory Data Analysis (EDA): Capture the profiling and analysis phase (data_manuplation/2_eda).
- Data Merging: Combine various datasets (data_manuplation/3_mergers and “include/c_data_merging”).
- Data Filtering: Filter and finalize the dataset (data_manuplation/4_filteringData and “include/d_data_filtering”).
- Machine Learning Modeling:
- Divide into subsections (present in the “ml_modeling” directory):
- Data Preparation (stage 1): Prepares the dataset with Spark and notebooks.
- Model Training:
- Naive training (2_naivetraining)
- Augmented training (3_augumented_training)
- Hyperparameter tuning (4_hptuning, with guidance from WandB as noted in the README)
- Final model selection and conversion to ONNX (5_selecting_best_model_to_onnx)
- Subcomponents for each model type: note that the system uses an RNN for sequential data (ball-by-ball info), a CNN for player stats, and a DNN for team stats; these are later combined in an ensemble architecture.
- Supporting Components:
- Utilities: Represent the “utils” folder which contains helper scripts (architecture, data utilities, model utilities, spark_utils).
- Experiment Tracking & Storage: Note the use of Weights & Biases and Hadoop HDFS (listed in the README).
- Visualization & Documentation: Include the “public” folder with drawings (architectureOverview.drawio, images for pipeline, tuning, results, win/loss examples).

##### Connection & Data Flow
- That Apache Airflow orchestrates the ETL tasks by connecting the ingestion modules to processing tasks (each staged as an independent Airflow task).
- How the output from data processing flows into multiple ML modules (RNN, CNN, DNN) and finally into the ensemble model.

##### Architectural Patterns & Design Principles
- Separation of concerns:
- Data extraction, preprocessing, merging, and filtering are clearly segmented in different directories and modules.
- Model training stages are modular, allowing for experimentation (naive training, augmented training, hyperparameter tuning).
- Use containerization:
- Docker encapsulating the entire application, ensuring consistency across development and deployment.
- Fault tolerance & scalability considerations through Airflow’s DAG orchestration, Apache Spark’s distributed processing, and modular design that allows for future enhancements (like live API integration).

##### Final Advice for the Principal Software Engineer
1. Data Ingestion and Preprocessing
2. ETL Pipeline orchestrated by Airflow
3. ML Modeling (with sub-blocks for each model type)
4. Deployment Environment (Docker, etc.)
5. Supporting modules (configs, utilities, experiment tracking)
- Clearly label every block and annotate with technology stacks and responsibilities.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources & Ingestion
    DS["Data Sources (Cricsheet & ESPN) - (include/a_data_sources)"]:::ingestion
    Airflow["Airflow Orchestration (t20.py) - (dags/t20.py)"]:::ingestion
    DS -->|"triggers"| Airflow

    %% ETL Pipeline
    Preproc["Data Preprocessing - (data_manuplation/1_preprocessing)"]:::etl
    EDA["Exploratory Data Analysis - (data_manuplation/2_eda)"]:::etl
    Merge["Data Merging - (data_manuplation/3_mergers)"]:::etl
    Filter["Data Filtering - (data_manuplation/4_filteringData)"]:::etl

    Airflow -->|"orchestrates"| Preproc
    Preproc -->|"processes"| EDA
    EDA -->|"combines"| Merge
    Merge -->|"finalizes"| Filter

    %% ML Modeling Pipeline
    ML_DP["Data Preparation (Spark) - (ml_modeling/1_data_preparation)"]:::ml
    Naive["Naive Training - (ml_modeling/2_naivetraining)"]:::ml
    AugTrain["Augmented Training - (ml_modeling/3_augumented_training)"]:::ml

    Filter -->|"feeds into"| ML_DP
    ML_DP -->|"initiates"| Naive
    Naive -->|"enhances"| AugTrain

    %% Branch: Model Types
    RNN["RNN Model (Sequential Data)"]:::ml
    CNN["CNN Model (Player Stats)"]:::ml
    DNN["DNN Model (Team Stats)"]:::ml
    Ensemble["Ensemble Model"]:::ml

    AugTrain -->|"trains"| RNN
    AugTrain -->|"trains"| CNN
    AugTrain -->|"trains"| DNN

    RNN -->|"merges"| Ensemble
    CNN -->|"merges"| Ensemble
    DNN -->|"merges"| Ensemble

    %% Post-modeling steps
    HPT["Hyperparameter Tuning (WandB) - (ml_modeling/4_hptuning)"]:::ml
    ModelSelect["Model Selection & Conversion (ONNX) - (ml_modeling/5_selecting_best_model_to_onnx)"]:::ml

    Ensemble -->|"optimizes"| HPT
    HPT -->|"selects"| ModelSelect

    %% Deployment
    Docker["Deployment (Docker Container) [Dockerfile, docker-compose.override.yml, environment.yml]"]:::deploy
    ModelSelect -->|"delivers"| Docker

    %% Supporting Components
    Config["Configuration & Environment (configs)"]:::support
    Utils["Utilities (utils)"]:::support
    Doc["Visualization & Documentation (public)"]:::support

    %% Dotted connections for supporting artifacts
    Config -.-> Preproc
    Config -.-> ML_DP
    Utils -.-> Preproc
    Utils -.-> ML_DP
    Doc -.-> Docker

    %% Classes definitions for colors and styles
    classDef ingestion fill:#FFD700,stroke:#B8860B,stroke-width:2px;
    classDef etl fill:#87CEFA,stroke:#1E90FF,stroke-width:2px;
    classDef ml fill:#98FB98,stroke:#2E8B57,stroke-width:2px;
    classDef deploy fill:#FFB6C1,stroke:#FF69B4,stroke-width:2px;
    classDef support fill:#FFFACD,stroke:#DAA520,stroke-width:2px;

    %% Click Events for Data Sources & Ingestion
    click DS "https://github.com/ravikumarchavva/cricket-prediction/tree/main/include/a_data_sources"
    click Airflow "https://github.com/ravikumarchavva/cricket-prediction/blob/main/dags/t20.py"

    %% Click Events for ETL Pipeline
    click Preproc "https://github.com/ravikumarchavva/cricket-prediction/tree/main/data_manuplation/1_preprocessing"
    click EDA "https://github.com/ravikumarchavva/cricket-prediction/tree/main/data_manuplation/2_eda"
    click Merge "https://github.com/ravikumarchavva/cricket-prediction/tree/main/data_manuplation/3_mergers"
    click Filter "https://github.com/ravikumarchavva/cricket-prediction/tree/main/data_manuplation/4_filteringData"

    %% Click Events for ML Modeling
    click ML_DP "https://github.com/ravikumarchavva/cricket-prediction/tree/main/ml_modeling/1_data_preparation"
    click Naive "https://github.com/ravikumarchavva/cricket-prediction/tree/main/ml_modeling/2_naivetraining"
    click AugTrain "https://github.com/ravikumarchavva/cricket-prediction/tree/main/ml_modeling/3_augumented_training"
    click HPT "https://github.com/ravikumarchavva/cricket-prediction/tree/main/ml_modeling/4_hptuning"
    click ModelSelect "https://github.com/ravikumarchavva/cricket-prediction/tree/main/ml_modeling/5_selecting_best_model_to_onnx"

    %% Click Events for Supporting Components
    click Config "https://github.com/ravikumarchavva/cricket-prediction/tree/main/configs"
    click Utils "https://github.com/ravikumarchavva/cricket-prediction/tree/main/utils"
    click Doc "https://github.com/ravikumarchavva/cricket-prediction/tree/main/public"
```
