Worth Reading

University Testing

A new study demonstrates that the Graduate Record Exam (GRE) is a poor predictor of performance in a psychology graduate program. Intelligence researchers Robert J. Sternberg and Wendy M. Williams compared GRE scores with grades, professors' rankings, and dissertation quality for 165 advanced degree candidates at Yale University. Consistent with Sternberg's "Triarchic Theory of Human Intelligence," they expected GRE scores to predict grades but not necessarily measures of creativity or practical performance.


In fact, the GRE had only a modest correlation with first year grades and no statistical significance for subsequent coursework. For other aspects of graduate school performance, "ratings of analytical, creative, practical, research, and teaching abilities by primary advisers and ratings of dissertation quality by faculty advisers," test scores were generally not useful.


Sternberg and Williams urge their colleagues to replicate their research at other institutions: "We believe that our results underscore the need for serious validation studies of the GRE, not to mention other admissions indexes, against measures of consequential performances, whether of students or of professionals. Psychologists need to apply the same standards of falsifiability in their admissions process as in their scientific work. Too often, we believe the use of a test becomes self-perpetuating, without serious attempts to verify its effectiveness. Our study suggests the need to reflect on the use of tests before they become firmly -- and, as it sometimes, seems, irrevocably -- entrenched."


- "Does the Graduate Record Examination Predict Meaningful Success in the Graduate Training of Psychologists?" American Psychologist, Vol. 52. No. 6, June 1997, pp. 630-641.


As computerized admissions and licensing exams spread rapidly (see Examiner, Spring 1997 and Fall 1996), more attention must be paid to the consequences of using the new testing technology, particularly gender and equity impacts. That is the message of Cleveland State University Professor Rosemary E. Sutton in a paper delivered at a recent American Education Research Association meeting and revised in a new journal article.


According to Prof. Sutton, ongoing research is necessary because: "technological innovations frequently have unforeseen social consequences . . . assessment innovations frequently have negative unintended, unpredictable consequences . . . a small achievement effect for a subgroup of students [on a high stakes test] may make a large impact on their lives . . . [and] we can only study the impact of high stakes testing under real high stakes testing conditions."


Based on a review of nearly 90 published studies, Prof. Sutton concludes, "Before we can make any substantiated claims that computerized tests are 'safe and effective' for all students, we need empirical research on the role of differential experiences, time limits, expectancies and adaptive test strategies, the public nature of computers in testing sties, and immediate feedback."


"Equity and High Stakes Testing: Implications for Computerized Testing," Equity & Excellence in Education, April 1997, pp. 5-15.