Penn Center for Learning Analytics Wiki - User contributions [en]

Socioeconomic Status

2023-06-01T21:52:16Z

Zhanlan:

Yudelson et al. (2014) [https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.659.872&rep=rep1&type=pdf pdf]

* Models discovering generalizable sub-populations of students across different schools to predict students' learning with Carnegie Learning’s Cognitive Tutor (CLCT)

* Models trained on schools with a high proportion of low-SES student performed worse than those trained with medium or low proportion
* Models trained on schools with low, medium proportion of SES students performed similarly well for schools with high proportions of low-SES students

Yu et al. (2020) [https://files.eric.ed.gov/fulltext/ED608066.pdf pdf]

* Models predicting undergraduate course grades and average GPA

* Students from low-income households were inaccurately predicted to perform worse for both short-term (final course grade) and long-term (GPA)
* Fairness of model improved if it included only clickstream and survey data

Yu et al. (2021) [https://dl.acm.org/doi/pdf/10.1145/3430895.3460139 pdf]
*Models predicting college dropout for students in residential and fully online program
*Whether the socio-demographic information was included or not, the model showed worse accuracy and true negative rates for residential students with greater financial needs
*The model showed better recall for students with greater financial needs, especially for those studying in person

Kung & Yu (2020)
[https://dl.acm.org/doi/pdf/10.1145/3386527.3406755 pdf]
* Predicting course grades and later GPA at public U.S. university
* Equal performance for low-income and upper-income students in course grade prediction for several algorithms and metrics
* Worse performance on independence for low-income students than high-income students in later GPA prediction for four of five algorithms; one algorithm had worse separation and two algorithms had worse sufficiency

Litman et al. (2021) [https://link.springer.com/chapter/10.1007/978-3-030-78292-4_21 html]
* Automated essay scoring models inferring text evidence usage
* All algorithms studied have less than 1% of error explained by whether student receives free/reduced price lunch

Queiroga et al. (2022) [https://doi.org/10.3390/info13090401 pdf]

* Models predicting secondary school students at risk of failure or dropping out
* Models achieved high performances with an AUC ROC higher than 0.90 and F1 higher than 0.88
* Equal performance for both genders students who participated in the educational system and completed their studies
* Students engaged in social welfare programs results in fewer problems in education
* The F1 score for social welfare program is 0.80, while for no social welfare program is 0, indicating bias toward social welfare program students
* The two most important features to predict students at risk in secondary school in Uruguay early are first-year primary school zones (rural or urban) and sixth-year assessment-based grouping

Socioeconomic Status

2023-06-01T21:43:28Z

Zhanlan:

Yudelson et al. (2014) [https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.659.872&rep=rep1&type=pdf pdf]

* Models discovering generalizable sub-populations of students across different schools to predict students' learning with Carnegie Learning’s Cognitive Tutor (CLCT)

* Models trained on schools with a high proportion of low-SES student performed worse than those trained with medium or low proportion
* Models trained on schools with low, medium proportion of SES students performed similarly well for schools with high proportions of low-SES students

Yu et al. (2020) [https://files.eric.ed.gov/fulltext/ED608066.pdf pdf]

* Models predicting undergraduate course grades and average GPA

* Students from low-income households were inaccurately predicted to perform worse for both short-term (final course grade) and long-term (GPA)
* Fairness of model improved if it included only clickstream and survey data

Yu et al. (2021) [https://dl.acm.org/doi/pdf/10.1145/3430895.3460139 pdf]
*Models predicting college dropout for students in residential and fully online program
*Whether the socio-demographic information was included or not, the model showed worse accuracy and true negative rates for residential students with greater financial needs
*The model showed better recall for students with greater financial needs, especially for those studying in person

Kung & Yu (2020)
[https://dl.acm.org/doi/pdf/10.1145/3386527.3406755 pdf]
* Predicting course grades and later GPA at public U.S. university
* Equal performance for low-income and upper-income students in course grade prediction for several algorithms and metrics
* Worse performance on independence for low-income students than high-income students in later GPA prediction for four of five algorithms; one algorithm had worse separation and two algorithms had worse sufficiency

Litman et al. (2021) [https://link.springer.com/chapter/10.1007/978-3-030-78292-4_21 html]
* Automated essay scoring models inferring text evidence usage
* All algorithms studied have less than 1% of error explained by whether student receives free/reduced price lunch

Queiroga et al. (2022) [https://doi.org/10.3390/info13090401 pdf]

* Models predicting secondary school students at risk of failure or dropping out
* Models achieved high performances with an AUC ROC higher than 0.90 and F1 higher than 0.88
* Equal performance for both genders students who participated in the educational system and completed their studies
* Students engaged in social welfare programs results in fewer problems in education
* The F1 score for social welfare program is 0.80, while for no social welfare program is 0, indicating bias toward social welfare program students
* The two most important features to predict students at risk in secondary school in Uruguay early are First-year primary school zones (rural or urban) and sixth-year assessment-based grouping

Socioeconomic Status

2023-06-01T21:41:20Z

Zhanlan:

Yudelson et al. (2014) [https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.659.872&rep=rep1&type=pdf pdf]

* Models discovering generalizable sub-populations of students across different schools to predict students' learning with Carnegie Learning’s Cognitive Tutor (CLCT)

* Models trained on schools with a high proportion of low-SES student performed worse than those trained with medium or low proportion
* Models trained on schools with low, medium proportion of SES students performed similarly well for schools with high proportions of low-SES students

Yu et al. (2020) [https://files.eric.ed.gov/fulltext/ED608066.pdf pdf]

* Models predicting undergraduate course grades and average GPA

* Students from low-income households were inaccurately predicted to perform worse for both short-term (final course grade) and long-term (GPA)
* Fairness of model improved if it included only clickstream and survey data

Yu et al. (2021) [https://dl.acm.org/doi/pdf/10.1145/3430895.3460139 pdf]
*Models predicting college dropout for students in residential and fully online program
*Whether the socio-demographic information was included or not, the model showed worse accuracy and true negative rates for residential students with greater financial needs
*The model showed better recall for students with greater financial needs, especially for those studying in person

Kung & Yu (2020)
[https://dl.acm.org/doi/pdf/10.1145/3386527.3406755 pdf]
* Predicting course grades and later GPA at public U.S. university
* Equal performance for low-income and upper-income students in course grade prediction for several algorithms and metrics
* Worse performance on independence for low-income students than high-income students in later GPA prediction for four of five algorithms; one algorithm had worse separation and two algorithms had worse sufficiency

Litman et al. (2021) [https://link.springer.com/chapter/10.1007/978-3-030-78292-4_21 html]
* Automated essay scoring models inferring text evidence usage
* All algorithms studied have less than 1% of error explained by whether student receives free/reduced price lunch

Queiroga et al. (2022) [https://doi.org/10.3390/info13090401 pdf]

* Models predicting secondary school students at risk of failure or dropping out
* Models achieved high performances with an AUC higher than 0.90 and F1 higher than 0.88
* Equal performance for both genders students who participated in the educational system and completed their studies
* Students engaged in social welfare programs results in fewer problems in education
* The F1 score for social welfare program is 0.80, while for no social welfare program is 0, indicating bias toward social welfare program students
* The two most important features to predict students at risk in secondary school in Uruguay early are First-year primary school zones (rural or urban) and sixth-year assessment-based grouping

Public or Private K-12 School

2023-06-01T21:33:54Z

Zhanlan:

Public or Private K-12 School

2023-05-29T20:08:37Z

Zhanlan:

Verdugo et al. (2022) [https://https://www.researchgate.net/profile/Jonathan-Vasquez-Verdugo/publication/359176069_FairEd_A_Systematic_Fairness_Analysis_Approach_Applied_in_a_Higher_Educational_Context/links/622ba9e89f7b324634245afa/FairEd-A-Systematic-Fairness-Analysis-Approach-Applied-in-a-Higher-Educational-Context.pdf pdf]
* An algorithm predicting dropout from university after the first year
* Several algorithms achieved better AUC and F1 for students who attended public high schools than for students who attended private high schools.

Queiroga et al. (2022) [https://doi.org/10.3390/info13090401 pdf]

* Models predicting secondary school students at risk of failure or dropping out.
* Models achieved high performances with an AUROC higher than 0.90 and F1-Macro higher than 0.88.
* Models achieve better results when new data comes from the secondary education period (e.g., model M2G1-UTU achieved a performance of 95%).
* First-year primary school zones (rural or urban) and sixth-year assessment-based grouping are two of the most important attributes of this model.

Public or Private K-12 School

2023-05-29T20:07:22Z

Zhanlan:

Verdugo et al. (2022) [https://https://www.researchgate.net/profile/Jonathan-Vasquez-Verdugo/publication/359176069_FairEd_A_Systematic_Fairness_Analysis_Approach_Applied_in_a_Higher_Educational_Context/links/622ba9e89f7b324634245afa/FairEd-A-Systematic-Fairness-Analysis-Approach-Applied-in-a-Higher-Educational-Context.pdf pdf]
* An algorithm predicting dropout from university after the first year
* Several algorithms achieved better AUC and F1 for students who attended public high schools than for students who attended private high schools.

Queiroga et al. (2022) <nowiki>[https://doi.org/10.3390/info13090401 pdf]</nowiki>

<nowiki>*</nowiki> Models predicting secondary school students at risk of failure or dropping out.

<nowiki>*</nowiki> Models achieved high performances with an AUROC higher than 0.90 and F1-Macro higher than 0.88.

<nowiki>*</nowiki> Models achieve better results when new data comes from the secondary education period (e.g., model M2G1-UTU achieved a performance of 95%).

<nowiki>*</nowiki> First-year primary school zones (rural or urban) and sixth-year assessment-based grouping are two of the most important attributes of this model.

At-risk/Dropout/Stopout/Graduation Prediction

2023-05-29T20:03:10Z

Zhanlan:

Kai et al. (2017) [https://www.upenn.edu/learninganalytics/ryanbaker/DLRN-eVersity.pdf pdf]
* Models predicting student retention in an online college program
* J48 decision trees achieved much lower Kappa and AUC for Black students than White students
* J48 decision trees achieved significantly lower Kappa but higher AUC for male students than female students
* JRip decision rules achieved almost identical Kappa and AUC for Black students and White students
* JRip decision trees achieved much lower Kappa and AUC for male students than female students

Hu and Rangwala (2020) [https://files.eric.ed.gov/fulltext/ED608050.pdf pdf]
* Models predicting if a college student will fail in a course
* Multiple cooperative classifier model (MCCM) model was the best at reducing bias, or discrimination against African-American students, while other models (particularly Logistic Regression and Rawlsian Fairness) performed far worse
* The level of bias was inconsistent across courses, with MCCM prediction showing the least bias for Psychology and the greatest bias for Computer Science
* Multiple cooperative classifier model (MCCM) model was the best at reducing bias, or discrimination against male students, performing particularly better for Psychology course.
* Other models (Logistic Regression and Rawlsian Fairness) performed far worse for male students, performing particularly worse in Computer Science and Electrical Engineering.

Anderson et al. (2019) [https://www.upenn.edu/learninganalytics/ryanbaker/EDM2019_paper56.pdf pdf]
* Models predicting six-year college graduation
* False negatives rates were greater for Latino students when Decision Tree and Random Forest yielded was used
* White students had higher false positive rates across all models, Decision Tree, SVM, Logistic Regression, Random Forest, and SGD
* False negatives rates were greater for male students than female students when SVM, Logistic Regression, and SGD were used

Christie et al. (2019) [https://files.eric.ed.gov/fulltext/ED599217.pdf pdf]
* Models predicting student's high school dropout
* The decision trees showed little difference in AUC among White, Black, Hispanic, Asian, American Indian and Alaska Native, and Native Hawaiian and Pacific Islander.
* The decision trees showed very minor differences in AUC between female and male students

Gardner, Brooks and Baker (2019) [[https://www.upenn.edu/learninganalytics/ryanbaker/LAK_PAPER97_CAMERA.pdf pdf]]
* Model predicting MOOC dropout, specifically through slicing analysis
* Some algorithms performed worse for female students than male students, particularly in courses with 45% or less male presence

Baker et al. (2020) [[https://www.upenn.edu/learninganalytics/ryanbaker/BakerBerningGowda.pdf pdf]]
* Model predicting student graduation and SAT scores for military-connected students
* For prediction of graduation, algorithms applying across population resulted an AUC of 0.60, degrading from their original performance of 70% or 71% to chance.
* For prediction of SAT scores, algorithms applying across population resulted in a Spearman's ρ of 0.42 and 0.44, degrading a third from their original performance to chance.

Kai et al. (2017) [https://files.eric.ed.gov/fulltext/ED596601.pdf pdf]
* Models predicting student retention in an online college program
* J-48 decision trees achieved much higher Kappa and AUC for students whose parents did not attend college than those whose parents did
* J-Rip decision rules achieved much higher Kappa and AUC for students whose parents did not attended college than those whose parents did

Yu et al. (2021) [https://dl.acm.org/doi/pdf/10.1145/3430895.3460139 pdf]
* Models predicting college dropout for students in residential and fully online program
* The model showed better recall for students who are under-represented minority (URM; not White or Asian), male, first-generation, or with greater financial needs
* Whether the socio-demographic information was included or not, the model showed worse accuracy and true negative rates for residential students who are under-represented minority (URM; not White or Asian), male, first-generation, or with greater financial needs
* Both accuracy and true negative rates were better for students who are first-generation, or with greater financial needs

Verdugo et al. (2022) [https://dl.acm.org/doi/abs/10.1145/3506860.3506902 pdf]
* An algorithm predicting dropout from university after the first year
* Several algorithms achieved better AUC and F1 for students who attended public high schools than for students who attended private high schools.
* Several algorithms predicted better AUC for male students than female students; F1 scores were more balanced.

Sha et al. (2022) [https://ieeexplore.ieee.org/abstract/document/9849852]
* Predicting dropout in XuetangX platform using neural network
* A range of over-sampling methods tested
* Regardless of over-sampling method used, dropout performance was slightly better for males.

Queiroga et al. (2022) [https://doi.org/10.3390/info13090401 pdf]
* Models predicting secondary school students at risk of failure or dropping out.
* Models achieved high performances with an AUROC higher than 0.90 and F1-Macro higher than 0.88.
* Models achieve better results when new data comes from the secondary education period (e.g., model M2G1-UTU achieved a performance of 95%).
* First-year primary school zones (rural or urban) and sixth year assessment-based grouping are two of the most important attributes of this model.