A cessation of measurement: Identifying test taker disengagement using response time (2024)

Products
Services
Solutions
Research
Resources
About us
Tour Map Growth

Collaborative for Student Growth

Book

A cessation of measurement: Identifying test taker disengagement using response time

Wise, S., & Kuhfeld, M. (2020). A cessation of measurement: Identifying test taker disengagement using response time. In M. Margolis, & R. Feinberg (Eds.), Integrating Timing Considerations to Improve Testing Practices, Routledge.

June 2020

By: Steven Wise, Megan Kuhfeld

Description

This chapter explores both what happens when test takers disengage and how this disengagement should be managed during scoring. The potency of an item response’s information depends on the strength of a positive relationship between response correctness and achievement level. This is a familiar concept in measurement: under classical test theory authors routinely remove items that do not exhibit positive item-total correlations during item analyses. Hence, the choice to include/exclude rapid guesses during scoring hinges primarily on whether measurement is intended to indicate as accurately as possible what the test taker knows and can do or if it represents an achievement hurdle that is considered the test taker’s responsibility to clear. Research using this capability has revealed that test takers generally disengage during only a portion of their items and that these disengaged responses reflect a momentary cessation of measurement. These findings underscore the essential role of test-taking engagement in our pursuit of valid scores.

View book on publisher's website

This book was published outside of NWEA. The full text can be found at the link above.

Topics:

Associated Research

Related Topics

Date

Journal article

Assessment in the time of COVID-19: Understanding patterns of student disengagement during remote low-stakes testing

This study compared the test taking disengagement of students taking a remotely administered an adaptive interim assessment in spring 2020 with their disengagement on the assessment administered in-school during fall 2019.

By: Steven Wise, Megan Kuhfeld, John Cronin

Topics: Equity, ,

2022

Journal article

Six insights regarding test-taking disengagement

There has been increasing concern about the presence of disengaged test taking in international assessment programs and its implications for the validity of inferences made regarding a country’s level of educational attainment. In this paper, the author discusses six important insights yielded by 20 years of research on this and implications for assessment programs.

By: Steven Wise

Topics: ,

2021

Journal article

The impact of disengaged test taking on a state’s accountability test results

This study investigated test-taking engagement on a large-scale state summative assessment. Overall, results of this study indicate that disengagement has a material impact on individual state summative test scores, though its impact on score aggregations may be relatively minor.

By: Steven Wise, Jonghwan (Jay) Lee, Sukkeun Im

Topics: Equity, ,

2021

Journal article

Variation in respondent speed and its implications: Evidence from an adaptive testing scenario

The more frequent collection of response time data is leading to an increased need for an understanding of how such data can be included in measurement models. Models for response time have been advanced, but relatively limited large-scale empirical investigations have been conducted. We take advantage of a large data set from the adaptive NWEA MAP Growth Reading Assessment to shed light on emergent features of response time behavior.

By: Benjamin Domingue, Klint Kanopka, Ben Staug, James Soland, Megan Kuhfeld, Steven Wise, Chris Piech

Topics: ,

2021

Journal article

A method for identifying partial test-taking engagement

This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously.

By: Steven Wise, Megan Kuhfeld

Topics: , ,

2021

Journal article

Comparing different response time threshold setting methods to detect low effort on a large-scale assessment

This study uses reading test scores from over 728,923 3rd–8th-grade students in 2,056 schools across the US to compare threshold-setting methods to detect noneffortful item responses. and so helps provide guidance on the tradeoffs involved in using a given method to identify noneffortful responses.

By: James Soland, Megan Kuhfeld, Joseph Rios

Topics:

2021

Technical brief

Comparability analysis of remote and in-person MAP Growth testing in fall 2020

How have COVID-19 school closures impacted student academic growth and achievement? New research using fall 2020 MAP Growth assessment data for 4.4 million students provides new insights, key findings, and actionable recommendations.

By: Megan Kuhfeld, Karyn Lewis, Patrick Meyer, Beth Tarasawa

Topics: , ,

2020

123…8›

View all publications