Yesterday, I mentioned in Part Three of the NEPC Report that use of VAM scores to evaluate teachers was unwarranted because the foundation upon which that concept is built is based upon unproven assumptions. And, the reason those assumptions are unproven is that the basic assumption in the research involved “random student assignment”, a concept that was not adhered to in the study that presupposes it.
Today, I learned that Pearson, the company that publishes and grades more standardized tests than any other in the world, acknowledges that , “Students Are Not Randomly Assigned to Teachers.” (page 7)
Find out what's happening in Kirklandwith free, real-time updates from Patch.
Pearson’s report states:
Find out what's happening in Kirklandwith free, real-time updates from Patch.
“Measuring teacher effectiveness under experimental conditions would be relatively straightforward. Under such conditions, students would be randomly assigned to classrooms, and then a pre-test/post-test design could be used with the teacher considered to be a treatment variable. Statistical tests regarding the significance of the variability of students’ scores between teachers would then be sufficient to make valid decisions about the effects of individual teachers, because it could be assumed that other confounding variables were randomly distributed across classrooms. In other words, the assignment of students to teachers would not advantage or disadvantage any particular teacher. Unfortunately, randomly assigning students to teachers for the purpose of measuring teacher effectiveness is rarely feasible (or ethical), and students are not randomly assigned to classrooms in practice. Therefore, teacher effectiveness must be estimated under less than ideal conditions.” (pages 7-8) (Emphasis added)
The Pearson report goes on to say:
“Because students are not randomly assigned to teachers, ….. Statistical models cannot easily separate compositional effects due to the clustering of students from teacher effects.” (page 8) (Emphasis added)
The factors that influence student assignment to teachers are many and varied. They are also an integral part of our educational system. For example, placement in elective classes, like foreign languages or music, create flights of students with higher achievement scores. That means that the remaining pool of students assigned to a class that same period of the day would be statistically lower achievers. That factor alone disqualifies the use of VAM for anything significant in HR decisions.