Utility of Complex Alternatives in Multiple-Choice Items: The Case of All of the Above

Abstract

This study investigated the utility of <em>all of the above</em> (AOTA) as a test option in multiple-choice items. It aimed at estimating item fit, item difficulty, item discrimination, and guess factor of such a choice. Five reading passages of the <em>Key English Test</em> (<em>KET</em>, 2010) were adapted. The test was reconstructed in 2 parallel forms: Test 1 did not include the abovementioned alternative, whereas Test 2, administered 2 weeks later, included such an alternative. The 2 tests, 32 items each, were administered to 142 high school third-graders. Results, analyzed through 3-parameter logistic model, indicated that the multiple-choice questions, including the alternative <em>all of the above</em>, were easier. Results also revealed that the option <em>all of the above</em> increased the guess factor. Because guess factor is a source of measurement error, it may threaten test validity and reliability.