cahlr/pybkt-examples

1. This repository is an experiment suite around the external pyBKT library, not a reusable application service. Each top-level Python file is a standalone analysis script that runs one Bayesian Knowledge Tracing variant and prints/compares metrics such as AUC and RMSE.

2. The main computational boundary is between the experiment scripts and the input datasets in data/. The scripts consume CSVs or text exports, transform them into the format expected by pyBKT, fit models, and evaluate predictions. There is no database, API, or long-running backend.

3. The top-level modeling scripts represent separate experiment variants:

- BasicForgets.py for standard BKT and a forgets extension

- KtIdem.py for item difficulty / template-based guessing effects

- KtPps.py for prior-per-student behavior

- ItemOrderEffect.py for order-sensitive learning effects

- ItemLearningEffect.py for item-specific learning effects

These are independent workflows rather than modules in a single runtime graph.

4. The data/ directory is the primary data subsystem. It contains a mix of raw inputs and derived artifacts:

- assistments-style CSV files such as as.csv, ct.csv, builder_train.csv, and builder_test.csv

- preprocessed versions of those files

- raw GLOPS problem-set text files under data/glops-exact/

- processed GLOPS CSVs under data/glops-exact-processed/

This suggests an offline preprocessing stage that converts source logs into analysis-ready tables.

5. The data preprocessing utilities are data/filter_glops.py and data/filter_nips.py. These are the only explicit transformation scripts in the tree and likely sit upstream of the model scripts, cleaning or filtering source datasets before model fitting.

6. The GLOPS dataset appears to be a major input path for the prior-per-student experiment. The raw .txt problem-set files are mirrored as processed .csv files, indicating a pipeline of raw export → normalization/filtering → per-problem-set model evaluation.

7. The Assistments and Cognitive Tutor files support the other experiments, especially item difficulty, item order, and item learning effects. The readme indicates that these scripts use template IDs or skill IDs as the key modeling dimension, so the effective data flow is from per-attempt student response rows into skill/template-specific BKT variants.

8. The output of the system is primarily research metrics and tables, not stored artifacts. Each script appears to compute model parameters and then compare predicted correctness against held-out responses using cross-validation or train/test splits, with results reported in the readme rather than written to a persistent results store.

9. The repo depends on a small scientific Python stack: numpy, pandas, requests, and scikit-learn, plus the external pyBKT package. Architecturally, this means all computation happens locally in Python, with no special runtime infrastructure beyond the installed library and the input files.

10. The important architectural diagram should show a simple offline research pipeline: source educational datasets in data/ feed preprocessing scripts, which feed one of several standalone model-evaluation scripts, which in turn invoke pyBKT to fit BKT variants and emit evaluation metrics and comparison tables.

11. There is no shared application state across experiments. The top-level scripts are effectively separate entry points that reuse the same modeling library and dataset families, so coupling is low except for common assumptions about student-response table formats and template/skill identifiers.

12. In short, this repository is a collection of reproducible modeling examples for educational data mining. Its structure is dominated by dataset preparation and experiment-specific analysis scripts, with pyBKT as the core engine and the data/ directory as the main integration boundary.