Examining the GRE: Myths, Misuses, and Alternatives

What is the GRE?

 

The Graduate Record Exam (GRE), a three part multiple-choice computerized exam created by the Educational Testing Service (ETS), is administered to more than 350,000 students per year and used by approximately 60% of American graduate programs. Designed to predict success in graduate school, the exam is composed of a Quantitative, Verbal, and Analytical section, each scored on a scale of 200 to 800. Recognizing the shortcomings of its exam, in October 2002 ETS replaced the analytical portion with a section in which test-takers write two timed essays in response to prompts.

Does the GRE accurately predict success?

 

Like other multiple-choice exams, the GRE does not accurately assess a test-taker's full potential for achievement in scholastic, professional, or personal endeavors, and limits access to graduate school for many individuals, particularly women, students of color, and non-traditional applicants.

Despite its primary purpose of predicting success in graduate school, a GRE score adds little useful information to a student's application - the test's own developer admits that undergraduate college grades do a better job of forecasting graduate achievement. ETS concedes "[T]he limitations of graduate school admissions tests in the face of the complexity of the graduate education process have long been recognized…[the] critical skills associated with scholarly and professional competence…are not currently measured by graduate admissions tests."1

The ability of the GRE to predict first-year graduate grades is incredibly weak, according to data from the test's manufacturer. In one ETS study of 12,000 test takers, the exam accounted for a mere 9% of the differences (or variation) among students' first-year grades.2 Undergraduate grades proved to be a stronger predictor of academic success, explaining 14% of the variation in graduate school grades. An independent non-ETS study found an even weaker relationship between test scores and academic achievement - just 6% of the variation in grades could be predicted by GRE scores.3

What do the GRE validity studies from major universities show?

 

In a Yale University review of the GRE's predictive ability for graduate students in its psychology department, test scores explained only 3% of the difference in grades during the first two years of study.4 The predictive power of the exam dropped to 1% when more comprehensive measures of performance (e.g. overall faculty ratings of the quality of students' performance and faculty evaluations of dissertation work) were considered. In fact, the correlation between GRE scores and graduate performance was negative for women in Yale's graduate psychology program.

At the University of Texas, one study showed that the GRE is not a valid predictor of success for the Graduate School of Social Work as measured by grades, degree completion, and fieldwork quality. Researchers concluded that eliminating the GRE from the selection process would not lead to lower quality students but would "open opportunities for many other individuals of professional quality."5

Researchers at Bowling Green State University hypothesized that individuals with high GRE scores would complete a graduate program in geology at a faster rate than their lower-scoring peers. Findings from the study revealed the opposite: high-scoring students took longer to complete the degree. Furthermore, undergraduate grades proved to be a better predictor of completing the geology graduate program than did test scores.6

What impact does the GRE have on educational equity?

 

As is the case with most other college admissions tests, large disparities in GRE scores exist between different groups of students. Despite their higher grades as undergraduate and graduate students, females score on average 20-30 points lower than males on each section of the exam. African-American, Latino, and, Native American students on average score lower than White and Asian American students. When test scores play a major role in admissions decisions such gaps create graduate student bodies that are disproportionately White and male.

At New York University, bilingual Hispanic doctoral students who scored low on the GRE did outstanding work on a comparable test written in Spanish. Researchers reasoned that because the culturally-laden language of the GRE can lead to score differences, it is crucial to explore alternatives to the GRE for bilingual students.7

One study in the Journal of Negro Education revealed that at the University of Florida Black graduate students with low GRE scores were compared to White peers with high test scores. The Black students attained higher GPAs during graduate study than did the Whites, leading researchers to conclude that the GRE is insensitive to how Black students' skills and resources translate into performance.8

The GRE is particularly susceptible to the influence of socioeconomic class. ETS' own research has shown a strong relationship between family background and test scores. One study of applicants who scored between 750 and 800 on the exam found that only 4% of these high-scoring test-takers had fathers who had not completed high school; around half had fathers with bachelor's degrees or more, and of these, a whopping 90% had fathers with graduate or professional degrees. When family income was held constant, most of the test score differences between races disappeared or shrank dramatically.9

How is the GRE misused?  

ETS guidelines specify: "A cutoff score based solely on GRE scores should never be used as the sole criterion for denial of admissions."10 Yet one ETS study revealed that only 10% of schools adhere to these guidelines, with almost 30% of those surveyed indicating they use a cutoff score and 10% recommending use of a cutoff.11 ETS has done little to curb such misuses. The use of cut scores is particularly dangerous to low-income individuals and students of color, who on average score markedly below affluent and White students. The Ford and Danforth Foundations tracked hundreds of students of color they had funded for graduate school and concluded: "Large numbers of [our] fellows might well have not been admitted to graduate programs if admissions committees had employed only numerical parameters in their assessment of student potential…More than half of the students who earned a doctorate would not have been admitted to graduate school in the first place if a GRE-Verbal score of 500 or more had been a criterion."12

Is the GRE coachable?  

The GRE can be conquered with tricks having nothing to do with the knowledge, persistence, thoughtfulness, and other qualities that are vital to graduate study and professional performance. One coaching book advises: "Taking the GRE is a game with its own rules, traps, and measures of success…How you do on the GRE is an indication of how well you play the game, but it is not an indication of how 'intelligent' you are, or what kind of student you will make."13

The exam's susceptibility to coaching undermines educational equity by advantaging students who can afford test prep materials - many of whom already score in the upper percentiles - over those who cannot. The most comprehensive coaching classes (which generally offer the greatest score gains) cost upwards of $1,000 or more. One coaching company claims its students gain on average 212 points on the GRE - a substantial advantage in the graduate school application process.14 While ETS asserts that the GRE is not coachable, it promotes its own materials: test takers can purchase a diagnostic service for $15, Preparing to Take the General Test for $18, or can use the free POWERPREP software package. While there are no independent studies on coaching's impact on the GRE, independent studies of coaching for the similar SAT exams demonstrate that coaching can improve scores (see FairTest's The SAT Coaching Cover-Up).

How does the GRE's computerized format affect test takers?  

In 1992 ETS began offering test takers the option of a computerized version of the GRE; in 1999 it eliminated the pencil-and-paper exam completely. Like any multiple-choice exam, the "computer adaptive" GRE General Test fails to measure complex abilities such as writing, critical thinking, and research skills. It instead rewards test-taking skills such as pacing and strategic guessing, perhaps to an even greater degree than paper-and-pencil formats.

After years of denial, the Educational Testing Service (ETS) has acknowledged that the computerized format of the Graduate Record Exam (GRE) mis-assesses some test-takers. ETS now admits that scores for students who carefully answer questions at the beginning of the exam but guess quickly at the end as they run out of time may be inaccurate. The same problem exists for those who answer difficult questions correctly but get generally easier ones wrong. The shift to computerized testing also resulted in price increases for test takers. Test takers must now pay $105, more than double the price of the exam before it was completely computerized. ETS also acknowledges that taking the exam more than once will help boost scores. The high cost of the GRE makes retesting prohibitive for lower-income students, adding to the advantage affluent individuals already have on the test.

What's the alternative?  

Several highly regarded graduate programs have proven that institutions can conduct a successful admissions process without the use of test scores. A brief list of such programs includes: Harvard Divinity School, Simmons College School of Social Work, Bank Street College School of Education, Pratt Institute School of Architecture, Brown University Division of Engineering, and Columbia University Masters of Science in Journalism. These examples are similar to the nearly 800 undergraduate institutions that admit a substantial number of freshman applicants without regard to SAT or ACT scores (click here).

Such institutions rely on the wealth of information that is already available through the application process, including application essays, undergraduate grades, faculty recommendations, writing samples, and personal interviews.

Average 1999-2000 GRE scores by Ethnicity and Sex

  Verbal Quanti-
tative
Analytical
Ethnic Group      
American Indian 453 485 513
Asian/Pacific Islander 480 610 571
African American or Black 392 419 427
Mexican American 428 479 488
Puerto Rican 402 475 465
Other Hispanic or Latino 437 487 491
White 490 547 572
       
Sex      
Female 461 537 551
Male 470 634 576
ALL TEST-TAKERS 465 578 562

Approximately 340,000 test-takers, 58% female
Source: ETS, Sex, Race, Ethnicity, and Performance on the GRE General Test 2001-2002

References
1 Enright, M. K. & Gitorner, D. (1989). Toward a description of successful graduate students. Princeton, NJ: Educational Testing Service.
2 Educational Testing Service. (1998). GRE Guide to the Use of Scores, 1998-1999. Princeton, NJ.
3 Morrison, T. & Morrison, M. (1995). A Meta-Analytic Assessment of the Predictive Validity of the Quantitative and Verbal Components of the Graduate Record Examination with Graduate Grade Point Averages Representing the Criterion of Graduate Success. Educational and Psychological Measurement, v. 55 (no. 2) pp. 309-316.
4 Sternberg, R. & Williams, W. (1997). Does the Graduate Record Examination Predict Meaningful Success in the Graduate Training of Psychologists? American Psychologist, v. 52 (no. 6), pp. 630-641.
5 Milner, M., McNeil, J. & King, S.W. (1984). The GRE: A Question of Validity in Predicting Performance in Professional Schools of Social Work. Educational and Psychological Measurement, vol. 44, pp. 945-950.
6 Onasch, C. (1994). "Undergraduate Grade Point Average and Graduate Record Exam Scores as Predictors of Length of Enrollment in Completing a Mater of Science Degree." ERIC Document No. 375 739.
7 Bornheimer, D.G. (1984). Predicting Success in Graduate School Using GRE and PAEG Aptitude Test Scores. College and University, v. 60 (no. 1) pp. 54-62.
8 Scott, R.R. & Shaw, M.E. (1985). Black and White Performance in Graduate School and Policy Implications For Using GRE Scores in Admission. Journal of Negro Education, v. 54 (no. 1), pp. 14-23.
9 Penncock-Roman, M. (1994). Background Characteristics and Futures Plans of High-Scoring GRE General Test Examinees, research report ETS-RR9412 submitted to EXXON Education Foundation, Princeton, NJ: Educational Testing Service.
10 Educational Testing Service. (1991). Guide to the use of the Graduate Record Examinations Program. Princeton, NJ.
11 Oltman, P.K. & Harnett, R.T. (1984). The Role of the GRE General and Subject Test Scores in Graduate Program Admission. Princeton, NJ: Educational Testing Service.
12 Hartnett, R. & Payton, B.F. (1977). Minority Admissions and Performance in Graduate Study: Preliminary Study of Fellowship Programs of the Ford and Danforth Foundations. New York: Ford Foundation.
13 Jacobson, R. L. (1993). "Critics Say Graduate Record Exam does not measure qualities needed for success and is often misused." The Chronicle of Higher Education, March, pp. 27-28.
14 Princeton Review website (2002). On-line at http://www.princetonreview.com/grad/testprep/testprep.asp?TPRPAGE=64&TYPE=GRE