Item characteristics and test-taker disengagement in PISA
If test-takers do not engage with the assessment, the reliability of test scores and
the validity of inferences about their proficiency may suffer. Test-taker disengagement
is particularly likely in low-stakes assessments and, according to prior research,
for certain types of students. But levels of engagement may also be related to aspects
that test developers can manipulate, such as item characteristics. This paper investigates
which item characteristics are associated with two indicators of test-taker disengagement,
rapid guessing and breakoffs, in an international assessment of reading. Analyses
of data from almost 500 000 students from 67 countries and economies that took part
in the 2018 Programme for International Student Assessment (PISA) show that rapid
guessing was observed mainly on simple multiple-choice questions. Breakoffs were more
likely in the presence of idiosyncratic selected-response formats, such as hot spot
or matching tasks. Both rapid guessing and breakoffs were more frequent on tasks involving
long and complex texts.
Available from April 12, 2024
In series:OECD Education Working Papersview more titles