Comparing the prediction performance of item response theory and machine learning methods on item responses for educational assessments

Jung Yeon Park, Klest Dedja, Konstantinos Pliakos, Jinho Kim, Sean Joo, Frederik Cornillie, Celine Vens, Wim Van den Noortgate

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

To obtain more accurate and robust feedback information from the students’ assessment outcomes and to communicate it to students and optimize teaching and learning strategies, educational researchers and practitioners must critically reflect on whether the existing methods of data analytics are capable of retrieving the information provided in the database. This study compared and contrasted the prediction performance of an item response theory method, particularly the use of an explanatory item response model (EIRM), and six supervised machine learning (ML) methods for predicting students’ item responses in educational assessments, considering student- and item-related background information. Each of seven prediction methods was evaluated through cross-validation approaches under three prediction scenarios: (a) unrealized responses of new students to existing items, (b) unrealized responses of existing students to new items, and (c) missing responses of existing students to existing items. The results of a simulation study and two real-life assessment data examples showed that employing student- and item-related background information in addition to the item response data substantially increases the prediction accuracy for new students or items. We also found that the EIRM is as competitive as the best performing ML methods in predicting the student performance outcomes for the educational assessment datasets.

Original languageEnglish
Pages (from-to)2109-2124
Number of pages16
JournalBehavior Research Methods
Volume55
Issue number4
DOIs
StatePublished - Jun 2023

Keywords

  • Background information
  • Educational assessment
  • Explanatory item response model
  • Item response theory
  • Machine learning
  • Prediction performance

Fingerprint

Dive into the research topics of 'Comparing the prediction performance of item response theory and machine learning methods on item responses for educational assessments'. Together they form a unique fingerprint.

Cite this