Maine Develops Good Assessment System

Status: 
Archived
Subject: 
K-12 Testing

Maine has taken what is both a modest and an exceptional approach to state assessment. The Maine assessment system limits the extent and stakes of testing, uses only constructed-response items, gathers information on the consequences of testing and on instructional practice, and requires pre-service education in classroom assessment for teachers. The result is a limited program of testing coupled with concern for improving assessment and teacher competence and for making good use of assessment information.

On many grounds, Maine's system rates well in light of the Principles and Indicators for Student Assessment Systems of the National Forum on Assessment. (FairTest is completing an analysis of all states in light of the Principles, thanks to a grant from the Joyce Foundation).

The Maine Educational Assessment (MEA) administers extended-open-response-item exams at grades 4, 8 and 11 in reading, writing, math, science, social studies, and arts and humanities, and in health in grades 4 and 8. Extended open response typically means paragraph-length responses. No multiple-choice items are used, largely, the state says, because of the negative impact these tests have on curriculum and instruction, a case they make to the public. The state also is shifting its scoring and reporting from norm-referenced to standards-referenced, in which scores are reported according to set levels of achievement, because of the harmful impact on teaching of norm-referenced testing.

The state has or is developing content and performance standards and curriculum frameworks. As new ones are completed, the exams will be modified to align with the standards. The state continues to investigate the possible future uses of performance assessments and portfolios. Concerns about the intensity of testing in grade 4 have recently led to modifications in that assessment, and more may ensue to make the exam more developmentally appropriate.

Items are written by committees of teachers. The state believes that the quality of items has improved as teachers have gained practice in writing. The committees address whether at least some items assess complex and critical thinking about content in each subject area.

In reading and math, the exams include a set of common items taken by every student at the grade level, thereby enabling individual scores to be obtained. For writing, each student responds to a common grade-level prompt. In other subjects, the exams are matrix sampled, with each student answering the questions on one two-item set out of 12 such sets, thereby producing only school-level scores. The state is considering administering common items, and hence obtaining individual scores, in science, social studies, and arts and humanities.

The assessments are intended for program evaluation and improvement and for holding schools accountable to the public. Students may obtain rewards or recognition. No other consequences are attached to the results. While public release of scores constitutes somewhat high stakes, the stakes are lower than in other states.

At each administration of the MEA, students also respond to one of two forms of a questionnaire, mostly about instructional practices. Teachers and administrators also answer questionnaires. Demographic and program enrollment data are obtained. Thus, the education department has at least some information against which test results can be analyzed in order to think about improving educational practice.

Assessment review involves teachers, administrators, state education department and university personnel, outside experts, community groups, and (via questionnaire results) students. The state legislature audits the program every four years. All the reviews consider the impact of assessment on curriculum and instruction.

Pre-service teachers are required to become knowledgeable in classroom assessment, performance assessment, and portfolios, and in integrating classroom assessment and instruction. No further education in this field is required of in-service teachers, but the state does offer workshops in these areas, as does the state university system. Administrators and other personnel have similar requirements and options.

State questionnaires periodically ask teachers and principals about professional practices and development needs in assessment and about existing professional development. The student surveys also include questions about classroom assessment practices, particularly assessments and scoring that are similar to the state exams.

Common exam items are used only once and then released to the public, together with examples of student work scored at different standards levels, to help the public understand the tests. The state hopes districts and schools analyze student performance on the items to help improve instruction.

Taken together, these practices suggest an effort to gather information in a relatively unobtrusive manner (three grades, some matrix sampling), with relatively low stakes. While the tests are intended to guide curriculum, the potential harmful consequences are also considered. To use assessment information well, the state gathers a variety of additional information. The state is also paying some attention to ensuring professional competence in assessment.

While the state does not have a separate bias review committee, the teacher committees that draft items use sensitivity and bias review guidelines, and the items also are reviewed by MEA staff.

Students with an Individual Education Plan (IEP) for special needs and students with limited English proficiency (LEP) may be allowed accommodations for taking the exams. IEP students may, as a last resort, be excluded from the assessment for all or some subjects. Students with LEP or IEPs are held to the same standards as are all other students.

LEP students may take tests in other languages (to be translated at the district level) or have the exams read to them (except for the reading test) to ensure students understand the items and are not confused by words or terms that are not relevant to what is being assessed. No alternate forms of assessment (e.g., portfolios) are used to meet the needs of students with LEP or IEP. The scores of all students who complete all the common sections of the exam are included in the state reports.

Further Improvement

It is difficult to determine the actual quality of the assessments from survey information. Do the tests reasonably represent the domains in each tested area, so that all important areas are examined? Do items intended to gauge higher order thinking or the ability to use knowledge in a domain actually measure what is intended? Does a near-exclusive reliance on medium-length responses (a paragraph) enable assessment of more complex content or ability to use knowledge? (Perhaps the contemplated portfolios will do that.)

And while the stakes from the state are relatively low, response to the FairTest survey does not make clear just how teachers do or do not use the results to modify curriculum and instruction. It is thus not certain as to whether there are harmful or beneficial consequences to curriculum or instruction or to individual students via district or classroom uses of test results.

Also, more rather than less matrix sampling may be warranted, and writing likely could be assessed more richly than by having each student respond to the same prompt. Finally, additional professional development is needed to strengthen teachers ability to use classroom assessments.

These and related issues should be addressed by the state, by educators, and by the public in Maine. That is the way to make a solid and reasonable state assessment system even better.

To order the Principles, go to our order form.