Lightweight Explainable Machine Learning for Early Detection of Academic Performance Risk among Students

Abstract

Predictive models are rarely used unless educators can understand them and if they can, they can intervene before students complete their education and get the signs of academic failure. Three classifiers were designed and evaluated in this study: logistic regression, random forest, and extreme gradient boosting (XGBoost) with publicly available UCI Student Performance dataset (math course, n = 395) to predict academic performance risk. The final grade of less than 10 out of 20 was defined as risk, and 32.9% of students were at risk. The models were fine-tuned with stratified five-fold cross-validation on 80% of the data, and then tested once on a 20% stratified hold-out set, using cost-sensitive learning techniques to address the class imbalance problem. All models discriminated well (hold-out ROC-AUC 0.933–0.941; recall 0.846–0.962) and differences among them were small when compared with sampling uncertainty for the first (only) period for which the grade was available. The discrimination of the first period grade alone was 0.698–0.713 ROC-AUC. Results from the SHAP analysis of the XGBoost model revealed that most important variable for making predictions was prior assessment performance, with a mean absolute SHAP value of about 27 times higher than the next important variable. The results indicate that the predictive signal of the early term grades is relatively high, and that the signal of behavioural and demographic variables is relatively low. They also show that any methodology used to make the model explainable should be used in tandem with careful consideration of ethics and fairness when implementing it into real world use.