High-Stakes Tests Do Not Improve Learning

Status: 
Archived
Subject: 
K-12 Testing

A FairTest analysis of state-level test results on the National Assessment of Education Progress (NAEP) shows that states with high-stakes tests perform worse, not better, on NAEP math and reading tests. This finding undermines the arguments of test proponents that high-stakes tests will result in improved learning. This view is found, for example, in the grading formula used in Quality Counts, the recent Education Week report, in which states received points for having high-stakes tests.

 

High Stakes Tests Do Not Improve Learning demonstrates that students were less likely to reach a level of "proficient" or higher on the NAEP math or reading tests in states which had mandatory high school graduation tests. Those states also had more students who failed to reach NAEP's "basic" level. In addition, such states were less likely to show statistically significant improvement in moving their students to the "proficient" level than were states without high school graduation tests.

 

Math Performance

The report compared the results from the NAEP 1996 grades 4 and 8 math tests with whether a state required its students to pass a test to obtain a high school diploma in 1994-95 and 1995-96. States with high-stakes graduation tests (HSGT) have fewer students who reach "proficient" and more students who fail to reach the "basic" level on NAEP math tests. (The four levels on NAEP assessments are "advanced," "proficient," "basic," and below "basic").

 

Forty-three states participated in the 1996 grade 4 NAEP math test. At the national level, the proportion of students reaching at least the "proficient" level was 20 percent. Of the 24 states in which 20 percent or more of the students reached "proficient" or higher, only five had a HSGT in 1994-96. Of the 19 states which had fewer than 20 percent of the students reach "proficiency," 11 had a HSGT. Rank-ordered, only two of the top 15 states had a HSGT.

 

On the other hand, of the 18 states which equalled or did worse than the national average of 38 percent in the proportion of students failing to reach "basic," 12 had a HSGT. But of the 25 states which had fewer than the national average of students falling below the "basic" level, only five had a HSGT.

 

Forty states participated in the 1996 grade 8 NAEP math test. Ranked by percentage of students reaching "proficient" or higher, none of the top 17 states had a HSGT. But of the 22 states scoring worse than the national average of students, 13 had a HSGT.

 

Similarly, among the 21 states equalling or exceeding the national average percentage of students scoring below "basic," 14 had a HSGT. None of the 19 states which did better than the national average at getting all students to "basic" had a HSGT.

 

Between the grade 4 and grade 8 tests, there were 14 cases of statistically significant improvement in state scores in math from 1992 to 1996. Of the 14, only 4 were from states with a HSGT, a lower proportion than the share of states which have HSGTs (17 of 50). This certainly does not support the claim that states with high-stakes tests will improve most quickly.

 

Reading Performance

Of the 19 states which had 28 percent or more students reach "proficient" or higher (the national average) on the 1994 NAEP reading test, two had a HSGT in 1993-94. None of the top ten states had a HSGT. Of the 20 states which performed below the national average, 14 had a HSGT.

 

On the other hand, of 17 states which did better than the national average of 41 percent of students not reaching "basic," only one had a HSGT. Of the 22 states which performed equal to or worse than the national average, 15 had a HSGT.

 

Conclusions

NAEP results do not support the claim that having high-stakes tests leads to higher educational quality than does the absence of such tests. Rather, it appears that proponents have based their rationale for high- stakes testing on ideology, not evidence. This false rationale, in turn, is being used to engineer a sweeping change in test use in the U.S. Currently, six more states plan to introduce graduation tests -- even though there is no evidence they will help improve student learning.

 

On all three NAEP exams, few or none of the states with HSGTs bettered the national average for proportion of students at the below "basic" level. This suggests that HSGTs do not lead to improvement for the lowest-performing students, contrary to claims of test proponents. The evidence also suggests such states do not improve faster than do states without HSGTs.

 

The limitations of test-driven curriculum and instruction are among the possible reasons test-driven reform does not succeed. Test-driven instruction tends to produce inflated scores, as happened with the well-known "Lake Wobegon" case of all states being "above average" on norm-referenced tests. Inflated scores caused by teaching to the test do not show up in independent measures such as NAEP. Additionally, states with HSGTs are more likely to rely on multiple-choice items and less likely to include open-ended items, according to FairTest's recent study of state testing programs, Testing Our Children, while NAEP tests contain a majority of open-ended items.

 

In sum, NAEP results show that test-driven change is a profoundly mistaken approach to improving schooling. States should look at the actual results of this approach, not the ideological claims of test- proponents, and draw the conclusion that fewer, lower stakes is the preferred approach.

 

-- For a full copy of the FairTest report, with tables of grade 8 math and grade 4 reading result a SASE ($.52) to High-Stakes Tests at FairTest; or look on our website at www.fairtest.org.

 

-- Click here to order Testing Our Children,