# probberechts/soccerdata architecture

> Architecture diagram of the GitHub repository probberechts/soccerdata, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/probberechts/soccerdata
- Repository: https://github.com/probberechts/soccerdata

## Overview

##### Project type and purpose
- SoccerData is a Python library of web‐scraping “scraper” classes that fetch soccer data from several external websites (Club Elo, ESPN, FBref, etc.) and return Pandas DataFrames with a unified schema.
- It is essentially an open‐source data‐acquisition tool, not a monolithic application or microservice.

##### File structure
- Top level:
- soccerdata/ — the core package
- docs/ — Sphinx documentation, notebooks, examples
- tests/ — pytest test suite, canned config and JSON fixtures
- config/build files: pyproject.toml, Makefile, noxfile.py, .github/workflows
- Within soccerdata/:
- _common.py, _config.py — shared base classes, configuration and caching logic
- clubelo.py, espn.py, fbref.py, … understat.py, whoscored.py — one module per scraper source
- Configuration & caching: uses local files under tests/appdata/config; likely a cache directory under the user’s home or temp.

##### README for additional insights
- Each scraper is a class (e.g. FBref) instantiated with league & season, exposing methods like read_schedule(), read_team_season_stats(), etc.
- Data is downloaded on demand and cached locally.
- API is synchronous, returns Pandas DataFrames.
- No centralized DB or message bus—external websites are the only “services.”

##### Details
- Caching strategy (e.g. “Checks FS cache before HTTP fetch”).
- Shared base class in _common.py as the abstraction point.
- Which config values live in _config.py vs. user‐exposed parameters.
- Don’t over‐modularize—group all 8+ external sites behind a single “External Services” node with a sub‐list.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Application
    Client["Client Application / Notebook"]:::support

    %% Library Runtime Components
    subgraph "Library Runtime" 
        direction TB
        subgraph "API Layer (Scraper Classes)" 
            direction TB
            APIGrp["API Layer"]:::runtime
            clubelo["clubelo.py"]:::runtime
            espn["espn.py"]:::runtime
            fbref["fbref.py"]:::runtime
            fotmob["fotmob.py"]:::runtime
            match_history["match_history.py"]:::runtime
            sofascore["sofascore.py"]:::runtime
            sofifa["sofifa.py"]:::runtime
            understat["understat.py"]:::runtime
            whoscored["whoscored.py"]:::runtime
        end
        Common["_common.py"]:::runtime
        Config["_config.py"]:::runtime
        Cache[(Local Cache / File System)]:::storage
    end

    %% External Services
    External[(External Websites)]:::external

    %% Supporting Artifacts
    subgraph "Supporting Artifacts"
        direction TB
        Tests["Testing Harness (pytest)"]:::support
        Docs["Documentation (Sphinx)"]:::support
        CI["CI/CD Workflows"]:::support
        Build["Build & Config Files"]:::support
    end

    %% Data Flow and Relationships
    Client -->|calls scraper| APIGrp
    APIGrp -->|inherits & uses| Common
    Config -->|provides endpoints & settings| Common
    Common -->|checks cache| Cache
    Cache -->|cache hit/miss| Common
    Common -->|HTTP fetch| External
    Common -->|returns DataFrame| Client

    Tests -->|uses fixtures & calls| APIGrp
    Docs -->|examples & reference| Client
    CI -->|runs tests & builds docs| Tests
    Build -->|project metadata & automation| CI

    %% Click Events
    click clubelo "https://github.com/probberechts/soccerdata/blob/master/soccerdata/clubelo.py"
    click espn "https://github.com/probberechts/soccerdata/blob/master/soccerdata/espn.py"
    click fbref "https://github.com/probberechts/soccerdata/blob/master/soccerdata/fbref.py"
    click fotmob "https://github.com/probberechts/soccerdata/blob/master/soccerdata/fotmob.py"
    click match_history "https://github.com/probberechts/soccerdata/blob/master/soccerdata/match_history.py"
    click sofascore "https://github.com/probberechts/soccerdata/blob/master/soccerdata/sofascore.py"
    click sofifa "https://github.com/probberechts/soccerdata/blob/master/soccerdata/sofifa.py"
    click understat "https://github.com/probberechts/soccerdata/blob/master/soccerdata/understat.py"
    click whoscored "https://github.com/probberechts/soccerdata/blob/master/soccerdata/whoscored.py"
    click Common "https://github.com/probberechts/soccerdata/blob/master/soccerdata/_common.py"
    click Config "https://github.com/probberechts/soccerdata/blob/master/soccerdata/_config.py"
    click Cache "https://github.com/probberechts/soccerdata/blob/master/tests/appdata/config/league_dict.json"
    click Cache "https://github.com/probberechts/soccerdata/blob/master/tests/appdata/config/teamname_replacements.json"
    click Docs "https://github.com/probberechts/soccerdata/tree/master/docs/"
    click Docs "https://github.com/probberechts/soccerdata/blob/master/docs/conf.py"
    click Docs "https://github.com/probberechts/soccerdata/tree/master/docs/reference/"
    click Tests "https://github.com/probberechts/soccerdata/tree/master/tests/"
    click Tests "https://github.com/probberechts/soccerdata/blob/master/tests/conftest.py"
    click Tests "https://github.com/probberechts/soccerdata/blob/master/tests/test_*.py"
    click CI "https://github.com/probberechts/soccerdata/blob/master/.github/workflows/ci.yml"
    click CI "https://github.com/probberechts/soccerdata/blob/master/.github/workflows/release.yml"
    click CI "https://github.com/probberechts/soccerdata/blob/master/.github/workflows/labeler.yml"
    click CI "https://github.com/probberechts/soccerdata/blob/master/.github/workflows/constraints.txt"
    click Build "https://github.com/probberechts/soccerdata/blob/master/pyproject.toml"
    click Build "https://github.com/probberechts/soccerdata/blob/master/noxfile.py"
    click Build "https://github.com/probberechts/soccerdata/tree/master/Makefile"
    click Build "https://github.com/probberechts/soccerdata/blob/master/.readthedocs.yml"

    %% Styles
    classDef runtime fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef external fill:#90EE90,stroke:#333,stroke-width:1px
    classDef storage fill:#90EE90,stroke:#333,stroke-width:1px
    classDef support fill:#FFA500,stroke:#333,stroke-width:1px
    class Client,Tests,Docs,CI,Build support
    class External storage,external
```
