Study: SAT Coaching Raises Scores

Status: 
Archived
Subject: 
University Testing

New research published in the Winter 2002 issue of Chance debunks test-maker claims that coaching for the SAT only minimally raises students’ scores. In “An SAT Coaching Program That Works,” Quinnipiac (Connecticut) University Professor of Mathematics Jack Kaplan describes a SAT-Math test prep class he led for two summers. The results indicate a substantial increase in SAT-M scores attributable to coaching.

 

Professor Kaplan’s research involved twenty-one students over two years, mostly from public, suburban schools near New Haven. Participants received 30-35 hours of instruction over the course of four weeks. Students sat for the SAT at least once before taking Professor Kaplan’s course, and at least once after the course concluded. Initial (pre-coaching) SAT-M scores ranged from 460 to 629 for the first round of participants and from 480 to 680 for the second year’s group. During the first year of the study, SAT-M scores of the participants increased by an average of 60 points. A different group of students posted average gains of 73 points in the second year. Students in a control group that did not participate in Kaplan’s class had average gains of only 13 points, near the national average, on a SAT-M retest.

 

The score gains attained by Professor Kaplan’s study far exceed the increases of 20-30 points the College Board cites as the average attributable to coaching. Although the College Board (which sponsors the SAT) sells its own test prep materials, it has long fought charges that intensive coaching can substantially raise test scores. One College Board student pamphlet states: “Longer programs (40 hours or more) improve scores an average of 15 to 20 points on verbal and 20 to 30 points on math…[B]eyond the first 20 or 30 hours of coaching, score gains are very minor.”

 

Professor Kaplan’s research led to a far different conclusion: “Even taking into consideration the small sample sizes and the possibility of some self-selection bias, this would seem to provide clear evidence that coaching on the math portion of the SAT’s can be far more effective than the College Board is willing to admit.” Twelve students earned SAT-Math scores that were 70 or more points higher from the pre- to post-test. Four increased their scores by 100-120 points, and one jumped a whopping 160 points. Particularly at selective colleges and universities, a 100-point SAT-Math score increase could tip the balance from a rejection to an offer of acceptance, debunking test-maker claims about the SAT’s value as a “common yardstick.”

 

While the small sample size limits generalizations from the study’s findings, the data are a clear challenge to College Board denials about the effectiveness of test prep. Since admissions offices have no way to discern which applicants were coached, Professor Kaplan’s study adds another reason for colleges and universities to question the role of test scores in their admissions processes.