# cahlr/pybkt-examples architecture

> Architecture diagram of the GitHub repository cahlr/pybkt-examples, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cahlr/pybkt-examples
- Repository: https://github.com/cahlr/pybkt-examples

## Overview

1. This repository is an experiment suite around the external `pyBKT` library, not a reusable application service. Each top-level Python file is a standalone analysis script that runs one Bayesian Knowledge Tracing variant and prints/compares metrics such as AUC and RMSE.

2. The main computational boundary is between the experiment scripts and the input datasets in `data/`. The scripts consume CSVs or text exports, transform them into the format expected by `pyBKT`, fit models, and evaluate predictions. There is no database, API, or long-running backend.

3. The top-level modeling scripts represent separate experiment variants:
   - `BasicForgets.py` for standard BKT and a forgets extension
   - `KtIdem.py` for item difficulty / template-based guessing effects
   - `KtPps.py` for prior-per-student behavior
   - `ItemOrderEffect.py` for order-sensitive learning effects
   - `ItemLearningEffect.py` for item-specific learning effects  
   These are independent workflows rather than modules in a single runtime graph.

4. The `data/` directory is the primary data subsystem. It contains a mix of raw inputs and derived artifacts:
   - `assistments`-style CSV files such as `as.csv`, `ct.csv`, `builder_train.csv`, and `builder_test.csv`
   - preprocessed versions of those files
   - raw GLOPS problem-set text files under `data/glops-exact/`
   - processed GLOPS CSVs under `data/glops-exact-processed/`  
   This suggests an offline preprocessing stage that converts source logs into analysis-ready tables.

5. The data preprocessing utilities are `data/filter_glops.py` and `data/filter_nips.py`. These are the only explicit transformation scripts in the tree and likely sit upstream of the model scripts, cleaning or filtering source datasets before model fitting.

6. The GLOPS dataset appears to be a major input path for the prior-per-student experiment. The raw `.txt` problem-set files are mirrored as processed `.csv` files, indicating a pipeline of raw export → normalization/filtering → per-problem-set model evaluation.

7. The Assistments and Cognitive Tutor files support the other experiments, especially item difficulty, item order, and item learning effects. The readme indicates that these scripts use template IDs or skill IDs as the key modeling dimension, so the effective data flow is from per-attempt student response rows into skill/template-specific BKT variants.

8. The output of the system is primarily research metrics and tables, not stored artifacts. Each script appears to compute model parameters and then compare predicted correctness against held-out responses using cross-validation or train/test splits, with results reported in the readme rather than written to a persistent results store.

9. The repo depends on a small scientific Python stack: `numpy`, `pandas`, `requests`, and `scikit-learn`, plus the external `pyBKT` package. Architecturally, this means all computation happens locally in Python, with no special runtime infrastructure beyond the installed library and the input files.

10. The important architectural diagram should show a simple offline research pipeline: source educational datasets in `data/` feed preprocessing scripts, which feed one of several standalone model-evaluation scripts, which in turn invoke `pyBKT` to fit BKT variants and emit evaluation metrics and comparison tables.

11. There is no shared application state across experiments. The top-level scripts are effectively separate entry points that reuse the same modeling library and dataset families, so coupling is low except for common assumptions about student-response table formats and template/skill identifiers.

12. In short, this repository is a collection of reproducible modeling examples for educational data mining. Its structure is dominated by dataset preparation and experiment-specific analysis scripts, with `pyBKT` as the core engine and the `data/` directory as the main integration boundary.

Main entry points: [`KtIdem.py`](https://github.com/cahlr/pybkt-examples/blob/master/KtIdem.py) (KtIdem), [`ItemOrderEffect.py`](https://github.com/cahlr/pybkt-examples/blob/master/ItemOrderEffect.py) (ItemOrderEffect), [`ItemLearningEffect.py`](https://github.com/cahlr/pybkt-examples/blob/master/ItemLearningEffect.py) (ItemLearningEffect), [`data/as.csv`](https://github.com/cahlr/pybkt-examples/blob/master/data/as.csv) (Assistments CSVs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data"]
  node_assistments["Assistments CSVs<br/>dataset<br/>[as.csv]"]
  node_ct["Cognitive Tutor CSV<br/>dataset<br/>[ct.csv]"]
  node_builder_train["Builder train<br/>dataset<br/>[builder_train.csv]"]
  node_builder_test["Builder test<br/>dataset<br/>[builder_test.csv]"]
  node_builder_train_pre["Train preprocessed<br/>dataset"]
  node_builder_test_pre["Test preprocessed<br/>dataset"]
  node_glops_raw["GLOPS texts<br/>dataset"]
  node_glops_processed["GLOPS CSVs<br/>dataset"]
  node_filter_glops["Filter GLOPS<br/>preprocessor<br/>[filter_glops.py]"]
  node_filter_nips["Filter NIPS<br/>preprocessor<br/>[filter_nips.py]"]
end

subgraph group_experiments["Experiments"]
  node_basic_forgets["BasicForgets<br/>experiment<br/>[BasicForgets.py]"]
  node_kt_idem["KtIdem<br/>experiment<br/>[KtIdem.py]"]
  node_kt_pps["KtPps<br/>experiment<br/>[KtPps.py]"]
  node_item_order["ItemOrderEffect<br/>experiment<br/>[ItemOrderEffect.py]"]
  node_item_learning["ItemLearningEffect<br/>experiment"]
  node_metrics["Metrics<br/>output"]
end

node_pybkt{{"pyBKT"}}

node_assistments -->|"raw rows"| node_filter_nips
node_ct -->|"raw rows"| node_filter_nips
node_builder_train -->|"preprocess"| node_builder_train_pre
node_builder_test -->|"preprocess"| node_builder_test_pre
node_glops_raw -->|"raw exports"| node_filter_glops
node_filter_glops -->|"writes"| node_glops_processed
node_filter_nips -->|"feeds"| node_basic_forgets
node_builder_train_pre -->|"feeds"| node_kt_idem
node_builder_test_pre -->|"evaluates"| node_kt_idem
node_glops_processed -->|"feeds"| node_kt_pps
node_assistments -->|"feeds"| node_item_order
node_ct -->|"feeds"| node_item_order
node_assistments -->|"feeds"| node_item_learning
node_ct -->|"feeds"| node_item_learning
node_basic_forgets -->|"fits"| node_pybkt
node_kt_idem -->|"fits"| node_pybkt
node_kt_pps -->|"fits"| node_pybkt
node_item_order -->|"fits"| node_pybkt
node_item_learning -->|"fits"| node_pybkt
node_basic_forgets -->|"reports"| node_metrics
node_kt_idem -->|"reports"| node_metrics
node_kt_pps -->|"reports"| node_metrics
node_item_order -->|"reports"| node_metrics
node_item_learning -->|"reports"| node_metrics

click node_assistments "https://github.com/cahlr/pybkt-examples/blob/master/data/as.csv"
click node_ct "https://github.com/cahlr/pybkt-examples/blob/master/data/ct.csv"
click node_builder_train "https://github.com/cahlr/pybkt-examples/blob/master/data/builder_train.csv"
click node_builder_test "https://github.com/cahlr/pybkt-examples/blob/master/data/builder_test.csv"
click node_builder_train_pre "https://github.com/cahlr/pybkt-examples/blob/master/data/builder_train_preprocessed.csv"
click node_builder_test_pre "https://github.com/cahlr/pybkt-examples/blob/master/data/builder_test_preprocessed.csv"
click node_glops_raw "https://github.com/cahlr/pybkt-examples/tree/master/data/glops-exact"
click node_glops_processed "https://github.com/cahlr/pybkt-examples/tree/master/data/glops-exact-processed"
click node_filter_glops "https://github.com/cahlr/pybkt-examples/blob/master/data/filter_glops.py"
click node_filter_nips "https://github.com/cahlr/pybkt-examples/blob/master/data/filter_nips.py"
click node_basic_forgets "https://github.com/cahlr/pybkt-examples/blob/master/BasicForgets.py"
click node_kt_idem "https://github.com/cahlr/pybkt-examples/blob/master/KtIdem.py"
click node_kt_pps "https://github.com/cahlr/pybkt-examples/blob/master/KtPps.py"
click node_item_order "https://github.com/cahlr/pybkt-examples/blob/master/ItemOrderEffect.py"
click node_item_learning "https://github.com/cahlr/pybkt-examples/blob/master/ItemLearningEffect.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_assistments,node_ct,node_builder_train,node_builder_test,node_builder_train_pre,node_builder_test_pre,node_glops_raw,node_glops_processed,node_filter_glops,node_filter_nips toneBlue
class node_basic_forgets,node_kt_idem,node_kt_pps,node_item_order,node_item_learning,node_metrics toneAmber
class node_pybkt toneNeutral
```

## Components

### Data

Raw and processed educational datasets plus preprocessing scripts.

- [Assistments CSVs](https://github.com/cahlr/pybkt-examples/blob/master/data/as.csv): dataset · Assistments-style student response tables for several experiments. · `data/as.csv`
- [Cognitive Tutor CSV](https://github.com/cahlr/pybkt-examples/blob/master/data/ct.csv): dataset · Cognitive Tutor response data used as an experiment input. · `data/ct.csv`
- [Builder train](https://github.com/cahlr/pybkt-examples/blob/master/data/builder_train.csv): dataset · Training split for Assistments-style modeling. · `data/builder_train.csv`
- [Builder test](https://github.com/cahlr/pybkt-examples/blob/master/data/builder_test.csv): dataset · Test split for Assistments-style evaluation. · `data/builder_test.csv`
- [Train preprocessed](https://github.com/cahlr/pybkt-examples/blob/master/data/builder_train_preprocessed.csv): dataset · Normalized training table for downstream model fitting. · `data/builder_train_preprocessed.csv`
- [Test preprocessed](https://github.com/cahlr/pybkt-examples/blob/master/data/builder_test_preprocessed.csv): dataset · Normalized test table for downstream prediction checks. · `data/builder_test_preprocessed.csv`
- [GLOPS texts](https://github.com/cahlr/pybkt-examples/tree/master/data/glops-exact): dataset · Raw problem-set exports used for prior-per-student experiments. · `data/glops-exact`
- [GLOPS CSVs](https://github.com/cahlr/pybkt-examples/tree/master/data/glops-exact-processed): dataset · Filtered and normalized GLOPS tables for analysis. · `data/glops-exact-processed`
- [Filter GLOPS](https://github.com/cahlr/pybkt-examples/blob/master/data/filter_glops.py): preprocessor · Converts raw GLOPS text exports into analysis-ready CSVs. · `data/filter_glops.py`
- [Filter NIPS](https://github.com/cahlr/pybkt-examples/blob/master/data/filter_nips.py): preprocessor · Cleans and reshapes source data for the experiment scripts. · `data/filter_nips.py`

### Experiments

Standalone Bayesian Knowledge Tracing variants and evaluation scripts.

- [BasicForgets](https://github.com/cahlr/pybkt-examples/blob/master/BasicForgets.py): experiment · Standard BKT and forgets variant benchmark. · `BasicForgets.py`
- [KtIdem](https://github.com/cahlr/pybkt-examples/blob/master/KtIdem.py): experiment · Item difficulty and template-based guessing experiment. · `KtIdem.py`
- [KtPps](https://github.com/cahlr/pybkt-examples/blob/master/KtPps.py): experiment · Prior-per-student experiment over GLOPS-style inputs. · `KtPps.py`
- [ItemOrderEffect](https://github.com/cahlr/pybkt-examples/blob/master/ItemOrderEffect.py): experiment · Order-sensitive learning effect experiment. · `ItemOrderEffect.py`
- [ItemLearningEffect](https://github.com/cahlr/pybkt-examples/blob/master/ItemLearningEffect.py): experiment · Item-specific learning effect experiment. · `ItemLearningEffect.py`
- **Metrics**: output · Reported AUC, RMSE, and comparison tables.

### Other components

- **pyBKT**: library · External Bayesian Knowledge Tracing engine used by all experiments.

## Connections

- Assistments CSVs → Filter NIPS: raw rows · Assistments-style tables are cleaned before modeling.
- Cognitive Tutor CSV → Filter NIPS: raw rows · Cognitive Tutor tables are cleaned before modeling.
- Builder train → Train preprocessed: preprocess · Training split is normalized for model input.
- Builder test → Test preprocessed: preprocess · Test split is normalized for evaluation.
- GLOPS texts → Filter GLOPS: raw exports · Raw GLOPS text files are converted to CSV.
- Filter GLOPS → GLOPS CSVs: writes · Preprocessing produces analysis-ready GLOPS CSVs.
- Filter NIPS → BasicForgets: feeds · Filtered student-response rows support the BKT baseline.
- Train preprocessed → KtIdem: feeds · Preprocessed builder training data supports item/template experiments.
- Test preprocessed → KtIdem: evaluates · Held-out builder data is used for prediction checks.
- GLOPS CSVs → KtPps: feeds · Processed GLOPS data drives prior-per-student modeling.
- Assistments CSVs → ItemOrderEffect: feeds · Assistments-style sequences support order-effect modeling.
- Cognitive Tutor CSV → ItemOrderEffect: feeds · Cognitive Tutor sequences support order-effect modeling.
- Assistments CSVs → ItemLearningEffect: feeds · Item-level response rows support learning-effect modeling.
- Cognitive Tutor CSV → ItemLearningEffect: feeds · Cognitive Tutor response rows support learning-effect modeling.
- BasicForgets → pyBKT: fits · Experiment scripts invoke the external BKT implementation.
- KtIdem → pyBKT: fits · Experiment scripts invoke the external BKT implementation.
- KtPps → pyBKT: fits · Experiment scripts invoke the external BKT implementation.
- ItemOrderEffect → pyBKT: fits · Experiment scripts invoke the external BKT implementation.
- ItemLearningEffect → pyBKT: fits · Experiment scripts invoke the external BKT implementation.
- BasicForgets → Metrics: reports · Outputs evaluation metrics and comparisons.
- KtIdem → Metrics: reports · Outputs evaluation metrics and comparisons.
- KtPps → Metrics: reports · Outputs evaluation metrics and comparisons.
- ItemOrderEffect → Metrics: reports · Outputs evaluation metrics and comparisons.
- ItemLearningEffect → Metrics: reports · Outputs evaluation metrics and comparisons.