ETS Cancels Faulty Computerized Test Scores

University Testing

The Educational Testing Service (ETS) has agreed to cancel unusually low computer-based Graduate Record Exam (GRE) scores reported to Amy Cuddy (see Examiner, Spring 1999) and to pay compensation for her time and registration fees.

Ms. Cuddy, now a graduate student in psychology at the University of Massachusetts-Amherst, had taken the computer-adaptive GRE in October 1998. Shocked by scores on the “Analytic” portion of the test which were several hundred points lower than on her practice exams, she signed up for one of the last administrations of the pencil-and-paper version of the GRE. Though ETS claimed the two test formats were comparable, Ms. Cuddy’s “Analytic” score soared from 300 (3rd percentile of test-takers) to 690 (84th percentile). Her GRE “Quantitative” score also increased significantly.

With free legal assistance from Jay Rosner, director of the Princeton Review Foundation, and guidance from FairTest, Ms. Cuddy sent a “demand letter” to ETS seeking cancellation of her score on the computer based GRE as well as compensation for out-of-pocket expenses. Under Massachusetts’ strong consumer protection law, ETS could either settle the claim or face court action in which it could be held liable for triple damages, attorneys fees and litigation costs.

ETS’s agreement to the demands indicates both flaws in its scoring formula for the computer-based GRE and an unwillingness to have these issues publicized in open court. Information exchanged in the course of settlement negotiations suggests that Ms. Cuddy’s low computer-based test scores reflected her pacing on the exam, not mastery of the material supposedly measured by the GRE. Critics have long argued that the forced-response format of ETS computer-adaptive exams (an item must be answered before the next one is displayed, and responses cannot be changed) puts a premium on test-taking skills not substantive knowledge.

The number of other test-takers unfairly disadvantaged by the computerized GRE is not known. Since the GRE is no longer available in paper-and-pencil format, future Amy Cuddys have no alternative.

For the fact sheet Computerized Testing: More Questions Than Answers, click here or send a self-addressed envelope with $.33 postage to FairTest.