"New" GRE Cancellation Reveals ETS Flaws

University Testing

The sudden decision by the Educational Testing Service (ETS) to cancel the long-delayed launch of a revised Graduate Record Exam (GRE) is the latest example of new exams being pushed toward the marketplace to increase test-maker revenues not improve assessment quality or meet students' needs.

ETS had claimed a GRE overhaul was necessary to measure students' preparation for graduate programs more accurately. The test-maker also expressed concern that some test items, which were repeatedly reused under the exam's computer-adaptive format, were being circulated on the internet and used by coaching schools. The "new" GRE was slated to be delivered via computer but each pool of questions would be used for only one day. As a result, the number of test administrations was going to be reduced to 30-35 each year. At present, the computer-adaptive exam is given several days each week year round at most U.S. test centers.

Worldwide registration bottlenecks were reported when ETS introduced a computerized Test of English as a Foreign Language (TOEFL) in early 2006. As a result, many overseas students could not take the test in time for application deadlines. The "new" GRE was scheduled to be implemented on the same Internet-Based Testing (IBT) platform as the TOEFL. ETS had already postponed the first administration of the revised GRE by a year in an attempt to buy time to upgrade its IBT network to ensure test-taker access (see Examiner, August 2006).

The decision leaves in place an exam which has many well-documented biases, weak predictive validity, and other flaws. The security problems caused by repeated reuse of the same items remain unaddressed.

Over the past decade, ETS has repeatedly tried to rush computerized exams into the marketplace before they were ready for "prime time." Among previous problems:

- More than 2,100 GMAT test-takers saw a "fatal error" warning pop up on their computer screens when they completed the GMAT and asked for score results in the summer and fall of 2001. ETS subsequently lost the contract to administer the GMAT (see Examiner, Fall 2001).

- Nearly 1,000 scores on the Graduate Management Admissions Test (GMAT) were miscalculated in 2000, but students were not informed of the mistake for almost a year, long after business schools had made decisions about their applications (see Examiner, Summer 2001).

- After the GRE completely eliminated pencil-and-paper testing in 1998-1999, a student won cancellation of her computerized scores by demonstrating that they were not comparable to her results on a prior version of the test. ETS admitted that the lower results stemmed from pacing of the electronic exam, not knowledge of the material. (see Examiner, Fall 1999).

- In December, 1997, the entire national computerized GMAT system crashed, leaving about 1,400 anxious business school applicants facing blank screens and forcing tests to be rescheduled. Many students who took early versions of the computerized GMAT and GRE reported facing a "black screen of death" during the test or when they asked for score results. No one knows whether their reported results were accurate.

- FairTest's GRE fact sheet is available here.