Study: Tests Don’t Improve Learning

Status: 
Archived
Subject: 
K-12 Testing

Powerful new evidence confirms FairTest’s long-standing claim that high-stakes standardized testing undermines rather than improves student learning and school quality. Calling high-stakes testing “a failed policy” with discriminatory impacts, Arizona State University (ASU) researchers Audrey Amrein and David Berliner conclude, “While a state’s high-stakes test may show increased scores, there is little support in these data that such increases are anything but the result of test preparation and/or the exclusion of students from the testing process.”

 

The authors examined 18 states that have implemented graduation exams and other high-stakes testing practices. The data used in the analysis consisted of scores obtained over two decades from four commonly used standardized tests: the ACT, SAT, National Assessment of Educational Progress (NAEP) and Advanced Placement (AP) tests.

 

The study investigated whether students demonstrated any transfer of knowledge beyond what was needed to perform on the state’s own high-stakes test. Testing, the authors point out, is intended to measure learning that goes far beyond the limited number of items on any particular exam. If scores on other tests increased following implementation of a high-stakes testing program, it would be evidence that this approach promotes transfer of learning.

 

The study found the policy “is not working,” concluding, “In all but one analysis...student learning is indeterminate, remains at the same level it was before the policy was implemented, or actually goes down when high-stakes testing policies are instituted.”

 

For example, average scores on the ACT, the college admissions exam most commonly used in states with high-school graduation exams, declined in 67% of the states requiring these exit tests. In addition, the percent of students taking the ACT grew more slowly in the high-stakes states than in the nation as a whole, suggesting to the researchers that the graduation tests are not encouraging more students to attend college. The results are similar for the SAT: 56% of the states with graduation tests experienced average SAT score declines, and SAT participation rates fell in 61% of the states when compared with the nation as a whole.

 

Overall, NAEP math and reading results at grades 4 and 8 had no correlation with the existence of high-stakes tests. (NAEP does not report state-level scores for grade 12). In reading, students in high-stakes states did improve slightly more from grade 4 in 1994 to grade 8 in 1998 than did the nation as a whole. But this was the only finding in the study to lend any support to proponents of high-stakes tests.

 

The researchers also found that states with high-stakes exams are more likely to exclude students with disabilities or limited English proficiency from participation in NAEP. This largely explained the greater-than-average score gains in the high-stakes states of Texas and North Carolina.

 

When rates of participation in the AP program were controlled, there was a decrease in the percentage of students scoring “3" or higher on AP tests in states with high-stakes graduation exams. Participation rates in AP fell in 67% of the high-stakes states, compared with the national average.

 

In sum, in states with high-stakes graduation tests, scores on independent exams provide no evidence of increased student learning and often decline relative to the nation as a whole, with many students apparently less well prepared and less likely to go to college than their peers in non-high-stakes states. The authors suggest this may be because high-stakes testing leads to narrow training to help students pass that specific test, not education that leads to genuine learning.

 

In part because states with high-stakes graduation tests are poorer and have larger proportions of minority-group students, the researchers point out that the damage (lower scores on most independent measures, lower rates of increase in college attendance) more often affects these students than their wealthier, majority-group peers. Thus, they conclude, “a high-stakes testing policy is more than a benign error in political judgment. It is an error in policy that results in structural and institutional mechanisms that discriminate against all of America’s poor and many of America’s minority students.”

 

• The report is in the peer-reviewed, scholarly journal Education Policy Analysis Archives available at http://epaa.asu.edu/epaa/v10n18/ and the Education Policy Studies Laboratory at ASU.