Predicting Secondary Student Academic Performance Using Stacked Regression Ensembles on UCI Datasets

Abstract

This study explores the use of stacked regression ensembles to predict secondary student academic performance using the UCI Portuguese student dataset. While previous works have focused on individual models such as Random Forest and XGBoost, this research investigates whether combining multiple regressors under a Theil-Sen meta-learner improves prediction accuracy. Among 15 models evaluated through cross-validation, XGBoost achieved the highest individual performance with R²  of 0.8420 and RMSE  of  1.2665. To further improve accuracy, stacking ensembles were created using 2 to 4 base regressors. The best-performing ensemble comprising XGBoost, ExtraTrees, LinearRegression, and Lasso achieved a cross-validated R² of 0.8553 and RMSE of 1.2082. These findings show that stacking diverse models offers enhanced predictive power and generalization, providing a robust solution for student performance prediction.