How cahlr/pybkt works
1. What this repository is
- pyBKT is a Python package for Bayesian Knowledge Tracing, with two interchangeable implementations:
- a pure Python stack under source-py/pyBKT
- a Python + C++ accelerated stack under source-cpp/pyBKT
- Both expose the same high-level API, so the packaging layer can choose the fastest available backend at install time.
2. Top-level architecture boundary
- The public entry point is the pyBKT package, centered on pyBKT.models.Model and pyBKT.models.Roster.
- Users interact with the model through fit, predict, evaluate, and crossvalidate.
- The rest of the repository is supporting infrastructure: data preparation, model generation, tests, and the C++ numerics backend.
3. Core domain model
- models/Model.py represents the trainable BKT model and its configuration.
- models/Roster.py extends the model into an online classroom-style state tracker for multiple students and skills.
- The model supports variants called out in the README: multiple guess/slip classes, multiple learn/forget classes, per-student or per-item parameters, and parameter fixing.
4. Training and inference pipeline
- fit/EM_fit.py coordinates parameter estimation using EM.
- fit/M_step.py handles the maximization step and parameter updates.
- fit/predict_onestep.py performs one-step-ahead prediction/evaluation after a model is trained.
- In the accelerated build, fit/E_step.cpp and fit/predict_onestep_states.cpp provide performance-critical inference logic.
5. Data ingestion and normalization
- The package accepts either Pandas DataFrames or CSV/TSV files.
- util/data_helper.py and util/check_data.py normalize input into the internal BKT data format and resolve column mappings via defaults.
- The model expects student response sequences with correctness values encoded as -1, 0, or 1, then transformed into the internal checkpoint matrix used by the EM algorithm.
6. Synthetic data and model generation
- generate/random_model.py, generate/random_model_uni.py, and generate/synthetic_data.py create synthetic BKT models and datasets for testing or experimentation.
- In the C++ tree, generate/synthetic_data_helper.cpp provides an optimized companion for data generation.
- This subsystem is separate from training: it produces inputs for the model rather than consuming them.
7. Evaluation and experiment utilities
- util/metrics.py provides supported scoring functions such as RMSE, AUC, and accuracy.
- util/crossvalidate.py wraps repeated fit/evaluate cycles for fold-based validation.
- util/dirrnd.py and util/print_dot.py are support utilities used by the training and reporting flow.
8. C++ backend structure
- source-cpp/pyBKT contains the accelerated implementation and a vendored copy of Eigen for linear algebra.
- The C++ code is focused on the computational hot spots of BKT rather than the whole API surface.
- The large Eigen/ subtree is a dependency bundle, not application logic; it supplies matrix, vector, and solver primitives used by the extension.
9. Package build and fallback strategy
- setup.py, setup.cfg, and MANIFEST.in define packaging.
- The installation path is intentionally dual-mode: if a native compiler/toolchain is available, pip installs the C++-accelerated version; otherwise it falls back to pure Python.
- This is an important architectural boundary because runtime performance and platform support depend on the install result.
10. Testing and examples
- The repository includes package-level tests in source-py/pyBKT/test and source-cpp/pyBKT/test.
- Tests cover training, prediction, hand-specified models, and dataset-specific behavior.
- The README’s usage examples mirror the actual package API and are useful for diagramming the external interface.
11. Execution flow from user input to output
- User supplies dataset and optional column mappings.
- Model.fit normalizes data, initializes parameters, and runs EM through the fit layer.
- The trained model stores learned priors, transitions, guesses, and slips.
- Model.predict and Model.evaluate reuse the trained parameters to produce predictions and metrics.
- Roster can then apply the trained model online to individual student trajectories.
12. Operational/CI boundary
- .github/workflows/publish.yml indicates automated packaging/publishing via GitHub Actions.
- This is a release/distribution concern rather than runtime logic, but it matters for how the package reaches PyPI and how the dual Python/C++ builds are distributed.
13. Best diagramming abstraction
- The cleanest architecture diagram is:
- User/API layer: Model, Roster, readme-exposed methods
- Data prep layer: file/DataFrame ingestion, defaults mapping, validation
- Learning/inference layer: EM fit, prediction, evaluation, cross-validation
- Acceleration layer: optional C++ extension
- Math dependency layer: Eigen
- Supporting tooling: generation, tests, packaging/CI
Main entry points: Model.py Model · EM_fit.py EM Fit · setup.py Setup · synthetic_data.py Synthetic data
Components 20
API
Public model interface and user-facing entry points
Learning
Parameter estimation, prediction, and validation
- EM Fit · Coordinates expectation-maximization training of BKT parameters
source-py/pyBKT/fit/EM_fit.py - M Step · Performs maximization and parameter updates during training
source-py/pyBKT/fit/M_step.py - Predict · Runs one-step-ahead prediction and evaluation after training
source-py/pyBKT/fit/predict_onestep.py - Cross-validate · Wraps repeated fit/evaluate cycles across folds
source-py/pyBKT/util/crossvalidate.py - Metrics · Implements RMSE, AUC, accuracy, and other evaluation measures
source-py/pyBKT/util/metrics.py
Data
Input normalization and data checks
- Data helper · Normalizes DataFrame and file inputs into internal BKT data structures
source-py/pyBKT/util/data_helper.py - Check data · Validates columns, defaults, and response encoding before training
source-py/pyBKT/util/check_data.py
Generation
Synthetic model and dataset generation
- Random model · Creates random BKT model instances for experiments and testing
source-py/pyBKT/generate/random_model.py - Random uni · Alternative random model generator with a simplified parameterization
source-py/pyBKT/generate/random_model_uni.py - Synthetic data · Generates synthetic student response datasets from BKT models
source-py/pyBKT/generate/synthetic_data.py
Backend
Optional C++ acceleration and numeric dependency
- C++ E-step · Native implementation of performance-critical EM inference logic
source-cpp/pyBKT/fit/E_step.cpp - C++ predict · Native implementation of one-step prediction state propagation
source-cpp/pyBKT/fit/predict_onestep_states.cpp - C++ synth helper · Native helper for faster synthetic data generation
source-cpp/pyBKT/generate/synthetic_data_helper.cpp - Eigen · Vendored matrix and solver dependency used by the accelerated extension
source-cpp/pyBKT/Eigen
Platform
Packaging, CI, and repository-level support
- Setup · Build and distribution configuration for selecting pure Python or C++ accelerated installs
setup.py - Manifest · Package inclusion rules and source distribution metadata
MANIFEST.in - Publish · Automated publishing workflow for package distribution
.github/workflows/publish.yml - Tests · Repository test suites covering training, prediction, and model behavior in both runtimes
source-py/pyBKT/test
Connections 24
- Model → Data helper: loads · Model accepts DataFrame or file input through normalization helpers
- Model → Check data: validates · Model uses data checks and column mapping defaults before training
- Model → EM Fit: fit · Model delegates parameter estimation to EM training
- Model → Predict: predict/evaluate · Model reuses trained parameters for inference and scoring
- Roster → Model: extends · Roster applies the model in an online multi-student setting
- EM Fit → M Step: updates · EM coordinates maximization step parameter updates
- EM Fit → C++ E-step: accelerates · Accelerated build can substitute native E-step logic
- Predict → C++ predict: accelerates · Accelerated build can substitute native prediction-state logic
- EM Fit → Metrics: evaluates · Training and validation consume scoring utilities
- Cross-validate → EM Fit: repeats · Cross-validation runs repeated fit cycles across folds
- Cross-validate → Metrics: scores · Cross-validation aggregates evaluation metrics
- Random model → Synthetic data: feeds · Generated models provide parameters for synthetic dataset generation
- Random uni → Synthetic data: feeds · Alternate model generator also feeds synthetic data creation
- Synthetic data → Tests: supports · Synthetic data generation aids testing and experimentation
- C++ synth helper → Synthetic data: accelerates · Native helper speeds up synthetic data generation
- C++ E-step → Eigen: depends on · Accelerated numerics rely on vendored Eigen primitives
- C++ predict → Eigen: depends on · Prediction-state computations rely on Eigen
- C++ synth helper → Eigen: depends on · Native generation helper uses Eigen-based numerics
- Setup → Model: packages · Packaging exposes the public Python API
- Setup → C++ E-step: builds · Install path can compile the accelerated extension when toolchains exist
- Setup → Publish: publishes · CI workflow automates release distribution
- Manifest → Setup: supports · Manifest constrains source distribution contents
- Tests → Model: verifies · Tests cover model fitting and behavior
- Tests → Predict: verifies · Tests cover prediction paths and outputs