How vingkan/haxml works
Project Type and Purpose
• This is an on-demand machine-learning prediction service (HaxML) for computing “expected goals” (XG) from gameplay data of the online game HaxBall.
• It combines data ingestion (scraping and downloading match data), offline analysis/modeling (feature engineering, training, evaluation), and an online serving layer (Flask API) that returns predictions to a frontend.
File-Structure Analysis
• Top-level directories:
• haxml/ ← core Python modules (utils, data inflation, prediction logic, evaluation, visualization)
• scripts/ ← data-download & preprocessing scripts (e.g., downloading match data from Firebase)
• notebooks/ ← Jupyter notebooks for exploratory data analysis, feature engineering, model comparison, visualization
• models/ ← serialized trained classifiers (.pkl files)
• server/ ← Flask application entry point for serving prediction and explanation endpoints
• Config & orchestration:
• Makefile ← reproducible data‐download and setup steps
• requirements.txt ← Python dependency list
• Procfile ← Heroku startup command (web: python server/__init__.py)
• .env (not committed) ← environment variables for Firebase credentials, port configuration
Key Insights from the README
• Data originates from a headless HaxBall API scraper, stored in Firebase in a “packed” schema, then inflated via haxml.utils.inflate_match.
• Offline pipeline: download → inflate → feature engineering → model training & evaluation → store .pkl models in models/.
• Serving pipeline: Flask app loads chosen model from models/, accepts HTTP requests, runs prediction code in haxml/prediction.py, returns XG probabilities.
• Deployment targets: local dev, Heroku (Procfile), or Digital Ocean droplet.
• Frontend integration: HaxClass Hub pages pass requests to HaxML endpoints when localml=true.
Main Components
a. Data Sources
• HaxBall headless API (online game room)
• Firebase real-time database (packed match data)
b. Offline Processing Pipeline
• Scripts (scripts/download_*.py) → download packed data
• Python modules (haxml/utils.py) → inflate and preprocess data
• Jupyter Notebooks → EDA, feature engineering, model training
• Model Registry (models/*.pkl) → persisted trained classifiers
c. Serving Layer
• Flask App (server/__init__.py)
• Prediction module (haxml/prediction.py)
• Evaluation module (haxml/evaluation.py) for introspection/explanation endpoints
d. Deployment & Infrastructure
• Local environment (virtualenv, .env)
• Build & orchestration (Makefile, Procfile)
• Hosting platforms: Heroku dynos, Digital Ocean droplet
e. Consumers / Frontend
• HaxClass Hub webpages (leaderboard.html, xg.html)
• HTTP client in browser
Architectural Patterns & Design Principles
• Separation of concerns: ingestion, preprocessing, modeling, serving
• Modular Python package (haxml/) used by both offline notebooks and online server
• Reproducible pipelines via Makefile/scripts
• Externalized configuration via .env and Procfile for 12-factor compliance
• Simple micro-service (Flask) for prediction serving
Final Checklist
• All major components present and labeled.
• Notes on protocols (HTTP, Firebase SDK).
• Deployment targets (Heroku/Digital Ocean) and how Procfile/Makefile coordinate startup.