Redux: Test Coaching Works

University Testing
Yet another study has demonstrated that test preparation can produce significant increases in SAT results, contrary to claims made by the exam's proponents. In "The Effectiveness of SAT Coaching on Math SAT Scores," published in Chance magazine (Vol. 18, No. 2, 2005), Quinnipiac College Professor of Mathematics Jack Kaplan reports on the experience of running his own SAT-prep. program. On average, his students raised their SAT Math scores by 78 points.


Kaplan's course includes 20 hours of classroom instruction and seven to 12 hours of individual tutoring. Of the 34 previously uncoached students he worked with over six summers, 21 increased their Math scores by more than 80 points from a baseline SAT administration with 11 going up by 100 points or more. Average score increases were higher for students with lower initial scores. In a control group Kaplan studied, average scores increased by only 15 points during the same period. Thus, he concludes that "the estimated coaching effect was 63 points" on the Math component of the SAT.


Based on his experiences and a review of the literature, Kaplan is critical of claims about the impact of coaching made by both the SAT's sponsor and major test preparation firms. He finds the College Board's long-standing assertion that short-term programs improve average math scores by about 15 points is "unsupportable to the point of being reckless." On the other hand, "commercial companies have made no serious effort to publicly substantiate the effectiveness of their courses" despite advertising meteoric score increases. More than fifteen years ago, a FairTest report, The SAT Coaching Cover-up, summarized published studies and concluded that well-crafted courses can raise scores by 100 points or more on the SAT Verbal plus Math 1600 point scale then in effect.


On balance, Prof. Kaplan concludes that "the effectiveness of SAT preparation programs remains very much an open question." He observes, "The College Board (and independent researchers who agree with them) point uncritically to studies that support their conclusions, paying little attention to those studies' flaws and limitations. Studies that do not support their conclusions are either ignored or explained away. They seem to be more concerned with defending the test against its critics than with providing accurate information."


Note that Prof. Kaplan is not affiliated with the coaching company with the same name. That commercial firm is a very profitable arm of the Washington Post corporation.