How doneria-anjali/student-performance-predictor works
1. Project purpose
- This repository is an offline educational-data-mining pipeline for comparing two prediction approaches on student learning data: Support Vector Machines (SVM) and Bayesian Knowledge Tracing (BKT).
- The core use case is to predict student performance from features such as knowledge components and step duration, using the KDDCup 2010 / Algebra 2005-2006 tutoring-system data.
2. Primary input data
- The algebra_2005_2006/ directory contains the raw dataset extracts:
- algebra_2005_2006.txt
- algebra_2005_2006_master.txt
- algebra_2005_2006_test.txt
- These appear to be the source records for training/testing and likely represent different splits or views of the same tutoring-system data.
3. Data preparation subsystem
- data-cleaning.py is the preprocessing stage.
- Its role is to transform the raw text dataset into an analysis-ready tabular form, producing clean_full_latest.csv.
- This is a clear architectural boundary: raw educational logs in, normalized CSV out.
4. Shared processed dataset
- clean_full_latest.csv is the central intermediate artifact.
- Both prediction implementations likely consume this cleaned dataset rather than reading the raw text files directly.
- In the architecture diagram, this file should be treated as the handoff point between preprocessing and modeling.
5. SVM prediction subsystem
- svm_student.py implements the SVM-based learner-performance predictor.
- The README indicates the model uses knowledge components and step duration as features, with the target being correctness of the attempt.
- This is the supervised-learning branch of the system.
6. BKT prediction subsystem
- bkt_method.py implements the BKT model.
- BKT is used to estimate latent skill mastery over time from student interaction sequences.
- This is the sequential / tutoring-model branch, distinct from the feature-based SVM approach.
7. Comparison-oriented architecture
- The repository is not an end-to-end product with a user interface; it is a research/experiment pipeline.
- The key architectural outcome is comparison of model behavior and prediction results between SVM and BKT on the same dataset.
- Any diagram should show both models fed from the same cleaned data source, with their outputs compared analytically.
8. Database and SQL layer
- database.py and sql_script.sql suggest a persistence layer used for loading, querying, or structuring the student data.
- This is likely auxiliary infrastructure for data management rather than the core modeling logic.
- If diagramming, place this as an optional supporting layer beneath preprocessing/modeling, not as the main runtime service.
9. Execution style
- The repository is Python-script driven, not service-based.
- The likely execution flow is batch-oriented: load raw data, clean it, run one or both models, inspect results.
- There is no evidence of a web server, API layer, or interactive application boundary.
10. Outputs and artifacts
- The main output artifacts are the cleaned CSV and model predictions/evaluation results generated by the Python scripts.
- adoneri_stdoshi_final_report.pdf is the documentation/report artifact, summarizing the experiment and findings rather than participating in runtime flow.
11. Important boundaries
- Boundary 1: raw educational logs vs cleaned features
- Boundary 2: feature-based prediction (SVM) vs mastery-tracking prediction (BKT)
- Boundary 3: analysis code vs optional persistence/SQL support
- These are the most important separations for an accurate architecture diagram.
12. Recommended diagram shape
- Show a left-to-right pipeline: raw Algebra/KDDCup data → cleaning/transformation → shared CSV/table → two parallel model branches (SVM and BKT) → prediction/evaluation comparison.
- Keep the database/SQL pieces as a supporting subsystem connected to the data preparation or storage stage, not the main application center.
Main entry points: data-cleaning.py Cleaner · clean_full_latest.csv Clean CSV · algebra_2005_2006 Algebra data · algebra_2005_2006.txt Raw logs
Components 15
Raw data
Algebra/KDDCup source extracts used as the common input corpus.
- Algebra data · Repository bundle of student interaction logs from the Algebra 2005-2006/KDDCup 2010 data.
algebra_2005_2006 - Raw logs · Primary raw text extract of the tutoring-system interactions.
algebra_2005_2006/algebra_2005_2006.txt - Master split · Master version of the Algebra 2005-2006 data used for training or consolidation.
algebra_2005_2006/algebra_2005_2006_master.txt - Test split · Held-out or alternate test view of the Algebra 2005-2006 data.
algebra_2005_2006/algebra_2005_2006_test.txt
Preparation
Cleaning and shaping stages that turn raw logs into analysis-ready data.
- Cleaner · Transforms raw tutoring logs into a normalized analysis-ready table.
data-cleaning.py - Clean CSV · Shared intermediate dataset consumed by both prediction branches.
clean_full_latest.csv - Feature view · Feature-oriented representation centered on knowledge components and step duration.
- Sequence view · Ordered student-action sequences used to estimate mastery over time.
Models
Parallel prediction approaches and their comparison outputs.
- SVM predictor · Feature-based classifier for predicting correctness from student activity signals.
svm_student.py - SVM results · Predictions and scoring from the SVM branch.
- BKT method · Bayesian Knowledge Tracing implementation for latent skill mastery estimation.
bkt_method.py - BKT results · Predictions and mastery estimates from the BKT branch.
- Compare models · Side-by-side comparison of SVM and BKT behavior on the same cleaned dataset.
Support
Auxiliary storage/query scripts that help manage the dataset.
- Database · Supporting data storage/query layer for structuring student records.
database.py - SQL script · SQL definitions or queries supporting dataset loading and organization.
sql_script.sql
Connections 18
- Algebra data → Raw logs: contains · The raw dataset directory bundles the main text extract.
- Algebra data → Master split: contains · The raw dataset directory includes the master split.
- Algebra data → Test split: contains · The raw dataset directory includes the test split.
- Raw logs → Cleaner: input · Cleaner ingests the raw student interaction log.
- Master split → Cleaner: input · Cleaner can also consume the master view of the data.
- Test split → Cleaner: input · Cleaner can also consume the test view of the data.
- Cleaner → Clean CSV: produces · Preprocessing emits the shared cleaned CSV artifact.
- Clean CSV → Feature view: derives · SVM-oriented tabular features are derived from the cleaned table.
- Clean CSV → Sequence view: derives · BKT-oriented interaction sequences are derived from the cleaned table.
- Database → SQL script: uses · Database helpers are backed by SQL definitions and queries.
- Cleaner → Database: loads · Preprocessing may use the persistence layer to stage or query records.
- SQL script → Clean CSV: supports · SQL layer supports shaping and extracting the cleaned dataset.
- Feature view → SVM predictor: features · Feature table feeds the SVM classifier.
- SVM predictor → SVM results: outputs · SVM branch produces predictions and evaluation artifacts.
- Sequence view → BKT method: sequence · Student interaction sequences feed the BKT model.
- BKT method → BKT results: outputs · BKT branch produces mastery estimates and predictions.
- SVM results → Compare models: compare · SVM results are compared against the alternative model.
- BKT results → Compare models: compare · BKT results are compared against the feature-based model.