# krishivgubba/bballrefapi architecture

> Architecture diagram of the GitHub repository krishivgubba/bballrefapi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/krishivgubba/bballrefapi
- Repository: https://github.com/krishivgubba/bballrefapi

## Overview

##### Project Type & Purpose
- The project is a public RESTful API designed specifically to provide basketball data scraped from basketball-reference.com.
- It uses Flask to serve endpoints, integrates with a PostgreSQL database (hosted on AWS RDS) for caching scraped data, and incorporates web scraping logic alongside additional features such as rate limiting and proxy rotation.
- It is intended for deployment (using Zeet) and local development.

##### Analyzing the File Structure
- The top-level files and directories include:
- Dockerfile, compose.yaml, and .dockerignore for containerization and deployment.
- Python modules (Player.py, Team.py, endpoints.py) that encapsulate the core functionality:
- Player.py contains the player scraping logic.
- Team.py handles team data scraping.
- endpoints.py creates the API endpoints with Flask.
- README files outlining usage and architecture considerations.
- The presence of __pycache__ and requirements.txt confirms a Python-based project with dependency management.

##### Main Components of the System
- Client: External consumers (e.g., frontend applications or other services) that will eventually call the API endpoints.
- API Server:
- Built using Flask, this server in endpoints.py handles incoming GET requests for various endpoints.
- Implements endpoints such as /api/player-stats, /api/team-year-roster, and /api/player-prev-games.
- Business Logic Modules:
- Player.py: Contains classes/methods for scraping and handling player data.
- Team.py: Manages team data scraping.
- Both modules include logic to decide whether to serve data from the database or invoke the scraper.
- Database:
- A PostgreSQL instance hosted on AWS RDS that caches scraped data for throttle control and efficiency.
- Web Scraping Component:
- Invoked when the requested data is not present in the database.
- Implements rate limiting (1 request per 5 seconds) and proxy rotation to avoid getting banned by basketball-reference.com.
- Deployment & Containerization:
- Dockerfile and compose.yaml indicate containerized deployment.

##### Relationships and Data Flow
- Clients make GET requests to the Flask API Server.
- The API Server processes these requests in endpoints.py and, based on the request, consults the Database.
- If data exists (“cache hit”), it is returned directly; if not (“cache miss”), the API Server calls the web scraping functions from Player.py or Team.py.
- New data scraped is written back to the Database for future requests.
- The web scraping component uses proxy rotation and respects rate limits as a safeguard.
- Interactions between these components to clearly detail the decision-making process for data retrieval.

##### Architectural Patterns & Design Principles
- Use of the “cache aside” pattern where the database is queried first and web scraping is only triggered on a cache miss.
- Separation of concerns (API layer in endpoints.py, scraping logic in separate modules) indicating modularity.
- Use of Docker for ensuring consistency across environments and the deployment pipeline through Zeet.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        Client("Client"):::client
    end

    %% API Layer
    subgraph "API Layer"
        API("Flask API Server"):::api
    end

    %% Business Logic
    subgraph "Business Logic"
        PlayerModule("Player Scraping Module"):::module
        TeamModule("Team Scraping Module"):::module
    end

    %% Data Layer
    subgraph "Data Layer"
        DB("PostgreSQL Database (AWS RDS)"):::db
    end

    %% Scraping Process
    subgraph "Scraping Process"
        Scraper("Web Scraping Service"):::scraper
        Proxy("Proxy Rotation Mechanism"):::proxy
        RateLimit("Rate Limiting (1req/5sec)"):::note
    end

    %% Deployment & Configuration
    subgraph "Deployment & Configuration"
        Zeet("Zeet Deployment"):::deploy
        Docker("Dockerfile"):::deployConfig
        Compose("compose.yaml"):::deployConfig
        DockerIgnore(".dockerignore"):::deployConfig
        Readme("README.Docker.md"):::deployConfig
        Dependencies("Dependency Management"):::dependency
    end

    %% Connections
    Client -->|"GET-request"| API
    API -->|"Query-cache"| DB
    DB -->|"Cache-hit"| API
    API -->|"Cache-miss(scrape)"| PlayerModule
    API -->|"Cache-miss(scrape)"| TeamModule
    PlayerModule -->|"InvokeScraper"| Scraper
    TeamModule -->|"InvokeScraper"| Scraper
    Scraper -->|"Uses"| Proxy
    Scraper -->|"StoreData"| DB
    Zeet -->|"Deploys"| API
    Dependencies --> API
    Dependencies --> PlayerModule
    Dependencies --> TeamModule
    %% Rate limiting annotation
    RateLimit -.->|"applies"| Scraper

    %% Click Events
    click API "https://github.com/krishivgubba/bballrefapi/blob/master/endpoints.py"
    click PlayerModule "https://github.com/krishivgubba/bballrefapi/blob/master/Player.py"
    click TeamModule "https://github.com/krishivgubba/bballrefapi/blob/master/Team.py"
    click Docker "https://github.com/krishivgubba/bballrefapi/tree/master/Dockerfile"
    click Compose "https://github.com/krishivgubba/bballrefapi/blob/master/compose.yaml"
    click DockerIgnore "https://github.com/krishivgubba/bballrefapi/blob/master/.dockerignore"
    click Readme "https://github.com/krishivgubba/bballrefapi/blob/master/README.Docker.md"
    click Dependencies "https://github.com/krishivgubba/bballrefapi/blob/master/requirements.txt"

    %% Styles
    classDef client fill:#d1c4e9,stroke:#512da8,stroke-width:2px;
    classDef api fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef module fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef db fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef scraper fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef proxy fill:#e2e3e5,stroke:#383d41,stroke-width:2px;
    classDef deploy fill:#ffcccb,stroke:#b22222,stroke-width:2px;
    classDef deployConfig fill:#ede7f6,stroke:#5e35b1,stroke-width:2px;
    classDef dependency fill:#ffe0b2,stroke:#fb8c00,stroke-width:2px;
    classDef note fill:#e0f7fa,stroke:#006064,stroke-width:2px;
```
