# vingkan/haxml architecture

> Architecture diagram of the GitHub repository vingkan/haxml, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vingkan/haxml
- Repository: https://github.com/vingkan/haxml

## Overview

##### Project Type and Purpose
- This is an on-demand machine-learning prediction service (HaxML) for computing “expected goals” (XG) from gameplay data of the online game HaxBall.
- It combines data ingestion (scraping and downloading match data), offline analysis/modeling (feature engineering, training, evaluation), and an online serving layer (Flask API) that returns predictions to a frontend.

##### File-Structure Analysis
- Top-level directories:
- haxml/            ← core Python modules (utils, data inflation, prediction logic, evaluation, visualization)
- scripts/          ← data-download & preprocessing scripts (e.g., downloading match data from Firebase)
- notebooks/        ← Jupyter notebooks for exploratory data analysis, feature engineering, model comparison, visualization
- models/           ← serialized trained classifiers (.pkl files)
- server/           ← Flask application entry point for serving prediction and explanation endpoints
- Config & orchestration:
- Makefile          ← reproducible data‐download and setup steps
- requirements.txt  ← Python dependency list
- Procfile          ← Heroku startup command (web: python server/__init__.py)
- .env (not committed) ← environment variables for Firebase credentials, port configuration

##### Key Insights from the README
- Data originates from a headless HaxBall API scraper, stored in Firebase in a “packed” schema, then inflated via haxml.utils.inflate_match.
- Offline pipeline: download → inflate → feature engineering → model training & evaluation → store .pkl models in models/.
- Serving pipeline: Flask app loads chosen model from models/, accepts HTTP requests, runs prediction code in haxml/prediction.py, returns XG probabilities.
- Deployment targets: local dev, Heroku (Procfile), or Digital Ocean droplet.
- Frontend integration: HaxClass Hub pages pass requests to HaxML endpoints when localml=true.

##### Main Components
a. Data Sources
- HaxBall headless API (online game room)
- Firebase real-time database (packed match data)
b. Offline Processing Pipeline
- Scripts (scripts/download_*.py)   → download packed data
- Python modules (haxml/utils.py)   → inflate and preprocess data
- Jupyter Notebooks                 → EDA, feature engineering, model training
- Model Registry (models/*.pkl)      → persisted trained classifiers
c. Serving Layer
- Flask App (server/__init__.py)
- Prediction module (haxml/prediction.py)
- Evaluation module (haxml/evaluation.py) for introspection/explanation endpoints
d. Deployment & Infrastructure
- Local environment (virtualenv, .env)
- Build & orchestration (Makefile, Procfile)
- Hosting platforms: Heroku dynos, Digital Ocean droplet
e. Consumers / Frontend
- HaxClass Hub webpages (leaderboard.html, xg.html)
- HTTP client in browser

##### Architectural Patterns & Design Principles
- Separation of concerns: ingestion, preprocessing, modeling, serving
- Modular Python package (haxml/) used by both offline notebooks and online server
- Reproducible pipelines via Makefile/scripts
- Externalized configuration via .env and Procfile for 12-factor compliance
- Simple micro-service (Flask) for prediction serving

##### Final Checklist
- All major components present and labeled.
- Notes on protocols (HTTP, Firebase SDK).
- Deployment targets (Heroku/Digital Ocean) and how Procfile/Makefile coordinate startup.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Sources
    subgraph "Data Sources"
        API["HaxBall API (Headless)"]:::external
        Firebase["Firebase DB"]:::database
    end
    API -->|"stores packed data via SDK"| Firebase

    %% Offline Processing Pipeline
    subgraph "Offline Pipeline"
        D1["Download Packed Matches"]:::script
        D2["Download All-Time Kicks"]:::script
        D3["Generate Match Metadata"]:::script
        Utils["Data Inflation & Utilities"]:::python
        Notebooks["Jupyter Notebooks"]:::notebook
        D1 --> Utils
        D2 --> Utils
        D3 --> Utils
        Utils --> Notebooks
        Notebooks -->|"trains & evaluates"| Models
    end

    %% Model Registry
    Models["Model Registry (models/*.pkl)"]:::models

    %% Serving Layer
    subgraph "Serving Layer"
        ClientReq["HTTP Client"]:::client
        Flask["Flask API Server"]:::service
        Predict["Prediction Module"]:::python
        Eval["Evaluation & Explanation Module"]:::python
        ClientReq -->|HTTP request| Flask
        Flask --> Predict
        Flask --> Eval
        Predict --> Models
    end

    %% Frontend
    subgraph "Frontend"
        Browser["HaxClass Hub Pages"]:::client
        Browser -->|XHR| Flask
    end

    %% Deployment & Orchestration
    subgraph "Deployment & Orchestration"
        Env[".env Config"]:::config
        Req["requirements.txt"]:::config
        Make["Makefile"]:::config
        Proc["Procfile"]:::config
        Env --> Make
        Make --> D1
        Make --> D2
        Make --> D3
        Make --> Notebooks
        Req -->|deps| Flask
        Proc --> Flask
    end

    %% Click Events
    click D1 "https://github.com/vingkan/haxml/blob/main/scripts/download_packed_matches.py"
    click D2 "https://github.com/vingkan/haxml/blob/main/scripts/download_all_time_kicks.py"
    click D3 "https://github.com/vingkan/haxml/blob/main/scripts/make_matches_metadata.py"
    click Utils "https://github.com/vingkan/haxml/blob/main/haxml/utils.py"
    click Predict "https://github.com/vingkan/haxml/blob/main/haxml/prediction.py"
    click Eval "https://github.com/vingkan/haxml/blob/main/haxml/evaluation.py"
    click Notebooks "https://github.com/vingkan/haxml/tree/main/notebooks/"
    click Models "https://github.com/vingkan/haxml/tree/main/models/"
    click Flask "https://github.com/vingkan/haxml/blob/main/server/__init__.py"
    click Req "https://github.com/vingkan/haxml/blob/main/requirements.txt"
    click Make "https://github.com/vingkan/haxml/tree/main/Makefile"
    click Proc "https://github.com/vingkan/haxml/tree/main/Procfile"

    %% Styles
    classDef external fill:#e2f0d9,stroke:#333,stroke-width:1px
    classDef database fill:#f9e79f,stroke:#333,stroke-width:1px
    classDef script fill:#d6eaf8,stroke:#333,stroke-width:1px
    classDef python fill:#d5f5e3,stroke:#333,stroke-width:1px
    classDef notebook fill:#f5b7b1,stroke:#333,stroke-width:1px
    classDef service fill:#fadbd8,stroke:#333,stroke-width:1px
    classDef client fill:#d6dbdf,stroke:#333,stroke-width:1px
    classDef models fill:#f9ebea,stroke:#333,stroke-width:1px
    classDef config fill:#e8daef,stroke:#333,stroke-width:1px
```
