Trusting Teacher Judgment

Status: 
Archived
Subject: 
K-12 Testing

Is teacher judgment a valid means of evaluating student learning? Will parents accept performance assessments and teacher judgments as valid evidence of learning? Can assessments rooted in teacher judgment be used in reporting school progress (“accountability”)? Research by University of Michigan professor Samuel Meisels and colleagues on the use of the Work Sampling System (WSS), which Meisels developed, suggest positive answers to the first two questions, thereby shedding light on the third.

 

The WSS is a curriculum-embedded, performance assessment system for students from pre-school to grade 5 used in many classrooms across the nation (see Examiner, Fall/Winter, 1995-96). Teachers collect and evaluate samples of student work in structured portfolios, use checklists, and prepare summary reports three times per year. Useful for helping teachers and for reporting to parents, the data are not intended for use in high-stakes decision-making or to compare students, but to monitor student progress in academic, personal and social domains.

 

The researchers compared WSS reports by 17 teachers with their students’ results on the individually administered Woodcock-Johnson (WJ) battery for literacy and math. They found correlations between WSS and the WJ comprehensive scores in reading, writing and math to range mostly from .50 and .75 . These are high enough to support a claim of teacher assessment accuracy and not so high as to suggest that the WSS only measures what the WJ measures. The WSS also was found to be an accurate means of identifying children in need of special services, based on correlations with the WJ.

 

A second study found that parents in Pittsburgh held positive attitudes toward the WSS and believed its use benefits their children whether they were high or low achievers. As with the previous study, most of the families were low-income African Americans. Parents who attended conferences with teachers were more likely to favor the WSS, with staff willingness to answer questions about the WSS rated very important. The more parents knew about WSS, the greater their satisfaction. The researchers concluded, “[T]his study demonstrates that when schools using a systematic, curriculum-embedded performance assessment make an effort to keep parents informed about the assessment, and when consistent informal communications between parents and teachers takes place, parental reactions to performance assessment can be very positive.”

 

If the WSS is accurate and parents respect it, could its results, such as teacher-generated summary reports, be aggregated for use in reporting student learning at the school level? The WSS was not designed for this purpose, and Prof. Meisels is concerned about the potentially corrupting effects on teacher judgments if they know the results could be used to evaluate schools or make high-stakes decisions about students. Nonetheless, in yet another study where the results of children who had been enrolled in WSS classrooms were compared with a demographically matched sample of children who had not experienced WSS and with all other children in the district, the WSS students showed substantially higher gains over time on conventional, group-administered achievement tests. The accuracy of this curriculum-embedded assessment and its acceptability to parents suggests that it might well be possible to construct public reporting systems based substantially on teacher judgment.

 

Similar findings of reliability and validity for the classroom-based Learning Record, including evidence of the accuracy of initial teacher judgments when Records are re-scored, also lends support to this approach to accountability (see Examiner, Summer 1995).

 

• “Trusting Teacher Judgments” will appear in the American Educational Research Journal; “
• "Parental Reactions to Authentic Performance Assessment” will appear in Educational Assessment.
• The third study is not yet ready for publication.
• The WSS is on the web at http://phcatalog.pearson.com/program_multiple.cfm?site_id=1021&discipline_id=3361&subarea_id=0&program_id=941; the Learning Record is on the web at http://www.cwrl.utexas.edu/~Syverson/olr/