The “New” SAT 2005: A Better Test or Just a Marketing Tool?

University Testing

In June 2002, the College Board announced a series of changes to the SAT-I that will be implemented in March 2005. The move primarily responds to threats that the University of California, the SAT’s biggest customer, might drop the test and to the growing number of colleges that have made test scores optional for many applicants.


Despite the slick packaging of the “new” test by the College Board, many questions and concerns remain unanswered. None of the revisions address the SAT-I’s fundamental flaws, such as the test’s inaccuracy, bias, and susceptibility to coaching. Nor has the College Board acted to crack down on widespread misuses of the SAT-I, such as requiring minimum scores for admissions or scholarships. Moreover, contrary to the marketing claims accompanying the promotion of the “new” SAT-I, the revised exam will do little to improve the quality of K-12 education.


How Will The “New” SAT-I Differ From The Current Test?
The overall format and content of the “new” SAT-I will remain largely unchanged from the current test. It will still be primarily multiple-choice and administered under strictly timed conditions. The SAT-Verbal will be renamed “Critical Reading” and will include additional short Reading Comprehension passages in place of much-criticized Analogy items. The math section will contain some Algebra II questions (it currently only covers Algebra I and geometry), and the arcane Quantitative Comparison items will be removed.


Responding to criticism about the SAT-I being far removed from classroom learning, the College Board will also add a so-called “Writing” section to the exam. Modeled after the current SAT II: Writing Test, the new section of the SAT-I will included 35 minutes of multiple-choice, copy editing questions and one short essay to be handwritten within a 25-minute time block. Each section will still be graded on a 200 to 800 point scale, so the addition of the third section will bump up a “perfect” SAT-I score to 2,400. The total testing time will rise from 3 hours to 3 ½ hours. These changes are expected to be accompanied by a cost increase of $10-$12.


Will The “New” SAT-I Predict College Grades More Accurately Than The Current Test Does?
College Board technical reports acknowledge that a student’s high school grades provide a better forecast of college performance than does the SAT-I. Since the revised SAT-I is still under development, there is no research demonstrating how the test’s “predictive validity” will be affected by the changes. However, several sources of information do provide clues about the changes’ likely impact.


With the exception of the added writing component, the “new” SAT-I will closely resemble the current test in form and content. This means the test will likely remain a weak predictor of college grades and bachelor degree attainment. Although the exam’s predictive validity may increase slightly due to the addition of a third section (predictive ability tends to rise with each additional test score), this improvement will probably not be substantial nor be equal across all demographic groups.


College Board research on the Test of Standard Written English (TSWE) it administered as part of the SAT prior to 1994, and the English Composition Achievement Test (ECT), which together were the prototypes for the current SAT II: Writing Test, calls into question the predictive power of the “new” SAT-I writing section. This research showed that the TSWE and ECT did a particularly poor job of predicting the college performance of African American students and students whose strongest language was not English. In fact, the TSWE was removed from the SAT in the early 1990s because the College Board recognized that it was not a useful tool in the college admissions process.


Will Changing The SAT-I Level The Playing Field For Students From Diverse Backgrounds?
College Board research demonstrates that the SAT-I systematically under-estimates the academic potential of young women, students whose home language is not English, and applicants over 25. This is unlikely to change with the “new” SAT-I. The underlying causes of the score gaps—including the test’s multiple-choice format, highly-speeded pace, and rewards for strategic guessing—will remain in place. The gender gap may be reduced slightly with the addition of the essay question, since females tend to score slightly higher than males on the SAT II: Writing Test, but will not be completely eliminated.


However, the score gap for students from non-English backgrounds will likely grow larger due to the added challenge from the high-pressure, timed conditions of the essay question. On the SAT II: Writing Test, African American and Latino test-takers score on average 80-100 points lower than White students. In fact, the SAT II: Writing Test has the second largest Black-White test score gap of the 12 most popular SAT II tests. These gaps will likely carry over to the “new” SAT-I, given that it will be nearly identical in form and content. Moreover, the College Board has made no indication it will act to stop test score misuses, such as minimum score cut-offs, that have a particularly harmful impact on the opportunities available to African-Americans, Latinos, low-income students, and students with special needs.


Will The “New” SAT-I Be Any Less Susceptible To Coaching?
While there is still debate over how much test prep can boost students’ scores, the College Board has backed away from its historic claim that the SAT-I is not “coachable” and now sells its own test preparation materials. Regardless of how much coaching increases SAT scores on average, it can substantially enhance some students’ results, thus further tilting the college admissions playing field.


The coaching industry is experiencing a boom in business from the coming SAT-I revisions. Firms such as Kaplan and the Princeton Review recognize that any change in the admissions process feeds student anxiety. They say the “new” SAT-I will be just as “coachable” as the current test, if not more so. One major test preparation company is touting its success in training test-takers to raise their SAT II Writing Test scores simply by adapting previously memorized essays. It concludes that adding the writing section to the SAT-I increases the exam’s coachability. The ability of coaching to boost students’ test scores skews the college admissions process in favor of students from higher-income families who can afford the $800 or more that an intensive course costs.


How Will The New SAT-I Essay Section Be Scored?
A proposed SAT-I essay section was scrapped in the early 1990s because of potential logistical problems in grading it and grave equity questions. These concerns have yet to be resolved. A June 2002 notice on the College Board’s website announced that SAT II: Writing Test scores would be delayed due to the unusually high volume of tests administered. Since there are close to ten times as many SAT-I’s administered as SAT II: Writing Tests, this experience calls into question the capacity to grade more than 2 million tests annually in a timely and fair manner.


Each essay will be read in less than four minutes and rated on a “holistic” 1-6 scale by two readers. If the readers’ scores differ by more than 2 points, a third person will score the essay to resolve the discrepancy. Readers will evaluate writing skills using a vague set of criteria, relying on the same guide now used to rate the SAT II: Writing Test. This guide emphasizes factors such as variety in sentence structure and range of vocabulary, which means test takers can earn high marks for complicated writing samples filled with “10-cent” words just as easily as they can for concise, interesting writing.


Will The Changes To The SAT-I Improve School Curriculum?
There is an old adage in the measurement profession: “What is tested becomes what is taught!” No matter how the SAT-I is altered, there will be strong pressure on teachers to drill their students on the narrow subject matter and formats it covers. One College Board study estimated that at least half of the high schools in the U.S. already offer SAT prep. Such practices will surely increase given the widespread anxiety among students and their parents about the “new” SAT.


Such drills come at the expense of more worthwhile learning opportunities. While the College Board boasts that the “new” SAT will provide an incentive for schools to teach writing skills, it will actually encourage educators to focus on how to write formulaic, five-paragraph essays rather than developing students’ communications skills more broadly. In addition, almost every state assessment system already includes a writing test, many of which allow students more time than the 25 minutes proposed for the SAT-I essay. Therefore, no additional “incentive” is needed in order to encourage teachers to focus on the limited writing skills covered by these assessments.


Why Should Any College Require The SAT-I, Old Or “New”?
More than 700 bachelor degree-granting institutions nationwide do not consider SAT-I or ACT scores before making admissions offers to substantial portions of their entering classes (see list here). They recognize that there is ample information in applicants’ files to make superior admissions decisions without the distortions caused by SAT-I scores. The “test-score optional” list already includes some of the most selective private colleges in the nation, such as Bates, Bowdoin, Mount Holyoke and, most recently, Sarah Lawrence, as well as large public campuses like the University of Texas at Austin. Rather than accepting the College Board’s promotional claims about the “new” SAT-I, colleges and universities should look with a critical eye at both the “new” and “old” exams and follow the lead of “test-score optional” institutions.