# panagiotisgia/arbitage_betting architecture

> Architecture diagram of the GitHub repository panagiotisgia/arbitage_betting, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/panagiotisgia/arbitage_betting
- Repository: https://github.com/panagiotisgia/arbitage_betting

## Overview

##### Main components
- Data Collection – Represent the web scraping modules that extract sports betting data from Novibet and Stoiximan websites. This includes:
- The web_scrape_functions module (with novibet_functions.py and stoiximan_functions.py) which uses Selenium for scraping.
- The web_scrape_functions_deb module (debugging notebooks for the same functions).
- Data Storage – The raw betting data saved as CSV files in the data directory.
- Data Processing and Analysis – Main processing components including:
- The main_scripts module that contains the main.ipynb notebook which orchestrates the process.
- The queries.py script which uses SQL queries to identify arbitrage opportunities based on the collected data.
- Configuration and Dependencies – files like requirements.txt provide information on the Python libraries, and a gitignore/LICENCE file manage project integration and licensing.

##### Relationships and interactions
- The Web Scraping layer (both production and debugging versions) extracts betting data from external websites.
- Once scraped, the data is written to CSV files in the data folder.
- The Main_scripts initiate the workflow by invoking the scraping functions and then processing the CSV files.
- The queries.py script then performs SQL queries on the processed data to detect arbitrage opportunities.
- The system flows from data extraction → data storage → data processing/querying → arbitrage output.

##### Architectural patterns and design principles
- Separation of Concerns: There is a clear partition between scraping (data ingestion), data storage (CSV files), and data processing (queries and calculation of arbitrage opportunities).
- Modularization: Distinct modules and folders (web_scrape_functions vs main_scripts) indicate modular coding for each domain.
- Extensibility: Debug notebooks (web_scrape_functions_deb) allow for rapid iterations and debugging, facilitating future enhancements.

##### Incorporate relevant technologies and tools
- Python (with Selenium for web scraping and data processing)
- SQL (for applying queries to detect arbitrage, likely via a Python library or an embedded database if adapted)
- Jupyter Notebooks (both for main orchestration and debugging)
- CSV data file format for temporary data storage.

##### Critical details
- Component names are clearly labeled.
- Debugging component (web_scrape_functions_deb) separately, but indicate its close relation with the main web scraping functions.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Production Web Scraping"
        NSF["Novibet Scraping Functions<br>(Uses Selenium)"]:::scraping
        SSF["Stoiximan Scraping Functions<br>(Uses Selenium)"]:::scraping
    end

    subgraph "Debug Web Scraping"
        NDN["Novibet Debug Notebook"]:::scraping
        SDN["Stoiximan Debug Notebook"]:::scraping
    end

    DS["Data Storage (CSV Files)<br>CSV output for scraped data"]:::storage
    MP["Main Processing Module<br>(Orchestration Notebook)"]:::processing
    SQ["SQL Query Engine<br>(queries.py)"]:::processing
    AR["Arbitrage Detector Output"]:::output

    subgraph "Configuration and Dependencies"
        Req["requirements.txt"]:::config
        Git[".gitignore"]:::config
        Lic["LICENCE"]:::config
        Read["README.md"]:::config
    end

    NSF -->|"writes_data"| DS
    SSF -->|"writes_data"| DS
    DS -->|"feeds_CSV_data"| MP
    MP -->|"executes_queries"| SQ
    SQ -->|"outputs_results"| AR

    %% Use one configuration file to indicate configuration dependency
    Req ---|"configures"| MP

    %% Click Events
    click NSF "https://github.com/panagiotisgia/arbitage_betting/blob/main/web_scrape_functions/novibet_functions.py"
    click SSF "https://github.com/panagiotisgia/arbitage_betting/blob/main/web_scrape_functions/stoiximan_function.py"
    click NDN "https://github.com/panagiotisgia/arbitage_betting/blob/main/web_scrape_funtions_deb/novibet.ipynb"
    click SDN "https://github.com/panagiotisgia/arbitage_betting/blob/main/web_scrape_funtions_deb/stoiximan.ipynb"
    click DS "https://github.com/panagiotisgia/arbitage_betting/tree/main/data/"
    click MP "https://github.com/panagiotisgia/arbitage_betting/blob/main/main_scripts/main.ipynb"
    click SQ "https://github.com/panagiotisgia/arbitage_betting/blob/main/main_scripts/queries.py"
    click Req "https://github.com/panagiotisgia/arbitage_betting/blob/main/requirements.txt"
    click Git "https://github.com/panagiotisgia/arbitage_betting/blob/main/.gitignore"
    click Lic "https://github.com/panagiotisgia/arbitage_betting/tree/main/LICENCE"
    click Read "https://github.com/panagiotisgia/arbitage_betting/blob/main/README.md"

    %% Styles
    classDef scraping fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef storage fill:#A9DFBF,stroke:#27AE60,stroke-width:2px;
    classDef processing fill:#F9E79F,stroke:#D4AC0D,stroke-width:2px;
    classDef output fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px;
    classDef config fill:#D5DBDB,stroke:#566573,stroke-width:2px;
```
