Testing Industry Critique Falls Short

K-12 Testing

A new think tank, Education Sector, launched itself earlier this year by publishing a report, Margins of Error: The Testing Industry in the No Child Left Behind Era. The study accurately analyzes the industry's serious limitations and the sad reality that most state exams overemphasize low-level skills and thinking, with harmful impacts on teaching and learning. Importantly, it calls for independent oversight of the testing industry. But it fails to address the concern that modest improvements in exam quality will not solve a larger educational problem: teaching to a test results in narrowing and dumbing-down curriculum.


In recommending "an independent national testing oversight agency" to be established by a "bipartisan presidential commission on standardized testing," author Thomas Toch harkens back to the report of the National Commission on Testing and Public Policy which issued a similar call more than a decade ago. FairTest from its inception has called for monitoring the testing industry, noting, "There is more regulation of pet food than of the tests that determine our childrens' futures."


Unanswered questions include whether Presidential appointees would select commissioners who would seriously question many politicians' favored "school reform strategies" and whether Congress would appropriate enough money for any commission to do an adequate job.


Better tests?

As an example of one of the better state exams, the Education Sector report highlights the Massachusetts Comprehensive Assessment System (MCAS), arguing Massachusetts has developed better tests because it spends more on their development than what the federal government pays states to comply with No Child Left Behind. But repeated reviews of the MCAS by parents, teachers and university faculty united in the Massachusetts Coalition for Authentic Reform in Education (CARE) found:


"In general, the conclusion was that the tests were very much like all the other standardized tests we have reviewed over the years; that they might be difficult for many students, not because they are particularly rigorous or challenging, but because they are long, tedious, lacking in a genuine performance base, and filled with ambiguities; and that the tests are likely to dampen student achievement by undermining quality.


"We were convinced that the tests are eminently coachable. In this regard, if the tests remain similar to what we reviewed, test scores are likely to improve without much attention by teachers to genuine content. We would be inclined, for example, to instruct students not to spend much energy reading the reading passages as many of the questions provide the content for the answers, and those that don't can be dealt with by a quick skim."


The tests were reviewable because Massachusetts is one of only a few states to release most test items each year. Maine, Ohio, Texas and New York are among the others.


CARE and others also reviewed other publicly disclosed MCAS tests, finding the history tests focused on arcane minutiae. For example, in one year the only question about the events leading up to the Civil War, the war itself, or its aftermath - clearly one the most critical periods in US history - asked the student to identify the union general in command at Gettysburg. In science, on one year's grade 10 test almost all the questions simply asked for the name of the concept. There was no way to know if the student understood the concept or its importance.


The first and second reviews:
[http://www.fairtest.org/MCAS%20Review%201.pdf and second year:


Other ostensibly praiseworthy tests have been similarly exposed. For example, teams of academics reviewed various New York Regents exams, finding them low-level, focused on trivia, and unrelated to doing college work. http://www.timeoutfromtesting.org/reports.php This may explain why, despite requests from the state, private schools in New York have universally refused to participate in the Regents program: they know it will dumb-down their curricula.


Ironically, a survey by Achieve, Inc., a group which promotes high-stakes testing, reinforces the conclusion that intensified testing is no solution to educational needs. Achieve surveyed professors who teach first year college students. The professors reported students had academic weaknesses in a variety of areas, including oral communication, understanding complicated reading materials, doing research, and producing quality writing. These areas cannot be addressed by testing, and the focus on testing is likely to ensure schools do not address them.


While Education Sector also recommends other changes, all but one of its proposals focus on developing better standardized exams. But if the nation is serious about high-quality education for all children, it cannot continue to mandate accountability programs tying high stakes to standardized tests. Aside from the irrationality of believing that high-quality teaching can be induced by threats, it simply is not feasible to assess in sufficient depth or quality what students should be learning.


An Alternative

There is a better solution, based on teachers assessing well in their classrooms (see article, this issue - Learning to Strengthen Formative Assessment Practices). As Black and Wiliam reported in 1998 in "Inside the Black Box" in Phi Delta Kappan, high-quality formative assessment can contribute mightily to improved learning outcomes, and it makes a greater contribution to improvement by low-achieving students than to high-achievers (see Examiner, Winter 1999).


Formative assessment does not include computerized multiple-choice mini-tests that are proliferating in big city districts and helping line the pockets of test producers. (Education Sector also recognizes the extremely low quality of these instruments, which are often labeled 'benchmark' assessments.)


Instead of wasting vast sums on standardized exams and test-prep materials, school systems, states and the federal government should support professional development that supports classroom, especially formative, assessment. To be truly effective, such programs must involve teacher collaboration; they must largely be under the control of teachers; and they should not be standardized (though teachers may at times choose to use standardized tools). High quality professional development and assessment are necessary and can be costly - but in the end, schools will have far better teachers. Combined with addressing wider issues facing schools and children, the nation could construct a far better school system. If, instead, systems keep throwing money at tests, the quality of teaching surely will not improve and is likely to decline, and with it, education.


- The report is at http://www.educationsector.org/research/research_show.htm?doc_id=346734