How jakekandell/nba-predict works
Main components and layers
• Data Layer – This includes raw input files (CSV files in the Data folder) as well as the source of NBA game data scraped from stats.nba.com.
• Scraping & Data Acquisition Module – Scripts such as getStats.py, availableStats.py, getDailyMatchups.py, and customHeaders.py are responsible for interfacing with external data sources, scraping relevant statistics, and setting up proper HTTP requests (possibly with custom headers). They feed the raw data into the processing pipeline.
• Data Preprocessing Layer – This comprises scripts that prepare the data for prediction. For example, standardizeStats.py and any custom processing done in configureCWD.py or teamIds.py are likely involved in cleaning, standardizing, and formatting data.
• Model Training & Persistence Layer – The createModel.py script is used for constructing and possibly training the logistic regression model, and the resulting model is saved in the SavedModels directory (finalized_model.pkl).
• Prediction & Execution Module – The nbaPredict.py and makePastPredictions.py scripts use the saved model along with fresh or historical game data to compute prediction outputs (win chances) for NBA games. These scripts also handle the generation and output of CSV files in the case of past predictions.
• Utility & Configuration – Files like requirements.txt ensure that the necessary libraries and dependencies are installed for everything to work smoothly.
Relationships and interactions among these components
• The Data Acquisition Module interacts with external data sources and outputs raw stats.
• Raw data is either directly used by the prediction modules or first passed through the preprocessing layer to standardize and format the data appropriately.
• Preprocessed data is then used both for training (when createModel.py is used) and for performing live or historical predictions.
• The final logistic regression model is stored persistently and later loaded by the prediction modules.
• The nbaPredict.py script is designed for daily/real-time prediction, while makePastPredictions.py is tailored for evaluating historical data over a set period.
• Utility scripts (e.g., configureCWD.py, teamIds.py) support both data acquisition and prediction logic by handling environment configuration and identification mappings.
Architectural Patterns and Design Principles
• Separation of Concerns: Different scripts are designed to handle distinct tasks such as data scraping, preprocessing, model creation, and prediction, ensuring modularity.
• Pipeline Architecture: The data flows sequentially from acquisition → preprocessing → model training → prediction, which should be clearly represented.
• Reusability and Persistence: The saved logistic regression model (finalized_model.pkl) emphasizes reusability, making it possible to separate the training process from daily predictions.
Key Technologies and Libraries
• Python is the primary programming environment.
• The project likely uses libraries for data scraping and HTTP requests (possibly requests or BeautifulSoup), data manipulation (like pandas), and machine learning (likely scikit-learn for logistic regression).
• The requirements.txt file would list these dependencies, which help in understanding the technology stack.
Specific Elements to be Included
• Clear labels for each component:
– “Data Preprocessing” (standardizeStats.py, customHeaders.py, configureCWD.py, teamIds.py)
– “Model Training & Persistence” (createModel.py and SavedModels/finalized_model.pkl)
– “Prediction Engine” (nbaPredict.py, makePastPredictions.py)
– External NBA statistics → Data Acquisition → Preprocessing → Model Training/Loading → Prediction