Some educators and corporate leaders would have us believe that the single most important factor influencing student achievement is the student’s teacher. They have launched a campaign to use student learning data to rid the schools of ineffective teachers.  There is no research evidence to support the claim that the teacher is the single most significant factor influencing student scores on end of year high stakes tests.

The Atlanta Journal-Constitution published two articles on teacher evaluation this week, and each of the authors, Jaime Sarrio, and Maureen Downey,  failed to include reference to published research on what influences student learning, and whether using student test scores is really a good idea in determining the difference between effective and ineffective teachers. As pointed out in the articles, Georgia received $400 million from the U.S. Department of Education’s Race to the Top Program and will implement a plan in which 50% of a teachers evaluation will be based on student test score data.

Using a model of assessment known as “Value-Added Modeling” (VAM) to measure student growth, a predicted score for a individual teacher is established based on student achievement gathered from previous years.  Once this trajectory or predicted score is known, comparisons can be made to determine effective and ineffective teachers.  But studies have shown that among teachers who scored in the the top 20% of effectiveness one year,  fewer than a third were in the top group the next year.  And another third of these effective teachers moved all the way to the bottom 40%.  As one researcher said, this runs counter to most people’s notions that the quality of a teacher is likely to change very little over time.  These research results raise serious questions about being able to measure the “teacher effect.”  If the students in the teachers class “do better” than the teachers’ VAM, the department of education would rate this teacher higher than a teacher whose student’s scores fall beneath his or her VAM. Sounds very straight forward, doesn’t it.

Bill Gates has a simpler idea. Measure the students on the way into class, then measure them again when they finish the course. From this you can calculate gain scores.  Bingo! You know if the teacher’s game is on. Neither of these ideas, using VAM or pre-post-testing has research evidence to back up the use of student test scores to make personnel, salary decisions or being able to differentiate between effective and ineffective teachers.

Every one of the hundreds of Georgia science teachers that I worked with in graduate programs at Georgia State University, and those teachers in whose schools I worked with in curriculum development projects, teacher education projects, and international environmental science projects know that what and how students learn is more complicated than what the Georgia Department of Education’s view on using student test scores to measure teacher effectiveness.

They know that the hundreds of students that they have taught come to school from real homes and families that vary considerably, and that many other out-of-school factors, which they can not control have a major influence on how students are motivated to learn, behave in school, get along with each other and are ready to learn. These professional teachers also know that many in school factors influence student learning.  The quality of leadership in their school is an important factor in creating learning environments that support teacher innovation and teacher decision making. And then there is the curriculum. Is the curriculum supportive of new research in the learning sciences that shows that student learning requires student to be the active participant in learning, and that their learning is largely dependent on teachers helping students make sense of their environment and construct meaning.  What a student learns in a science class, for instance, is somewhat dependent on the student’s active participation in learning.  In the 21st Century, the role of the teacher is to facilitate student learning, not pass information onto to students as if they were passive receptors of knowledge and information.

And then there is the issue of teacher and administration collaboration. Research into schools that foster collaboration among teachers, within a subject area or across subjects leads to a more professional work environment in that teachers with diverse professional experiences and expertise share with each other best practice, and discuss among their collaborative teams issues specific to curriculum.  In instances where there is a high degree of teacher collaboration, student test scores appear to improve. But collaboration might not be seen as important now that the State wants to weed out the “ineffective” teachers by using individual test scores posted by individual teacher’s students.

The quality of the teacher in the classroom is surely a significant and important factor influencing student learning.  But it is only one of many in-school and out-of school factors.  Researchers estimate that about 1/3 of student learning is influenced by in-school factors, and the quality of the teacher in the classroom is only one of at least four factors.  At most, the teacher on a good day might influence 1/3 of students learning growth, but in reality is more likely shared with other in-school factors including the quality of leadership, the nature of the curriculum, and degree of teacher collaboration.

Rating teachers using report cards and methods that rely on testing is a dangerous path. The research is not supportive on the use of use of student test scores to evaluate teachers. In fact the Board on Testing & Evaluation of National Academy of Sciences sent a letter and report to the US Department Education prior to release on the Race to the Top guidelines that States used to submit their proposals.  The Board stated in its letter: “VAM estimates of teacher effectiveness should not be used to make operational decisions because such estimates are far too unstable to be considered fair or reliable.”

Instead,  the Race to the Top officials insisted that student test data be used in any model of teacher evaluation. They went ahead with this mandate even after receiving Board on Testing & Evaluation letter and supporting research.

Should teachers be evaluated?  Of course.  But as in the business world, a more comprehensive, and holistic approach needs to be used to evaluate teachers.  There are much better ways to evaluate teacher performance that are grounded in research on teaching and learning.  Many school districts have worked with university researchers to develop systematic observation protocols with well developed, research based criteria to observe and examine teaching.  In addition to classroom observations, competent supervisors and peers can evaluate  teachers using videotapes of classroom teaching, teacher interviews, and artifacts of teaching such as lesson plans, assignments, teacher websites, and samples of student work.  Using this approach, a body of evidence is developed that gives a more comprehensive picture of a teachers professional work.

We need to be careful not to fall into the trap advocated by educators such as Joel Klein (former chancellor of N.Y.C. schools, and Michelle Rhee (former superintendent of the D.C. schools), and corporate heads such as Bill Gates that the single most important thing we can do to improve schools is removing incompetent teachers.  No one would question that we would hope that our students’ teachers are competent—indeed, outstanding.  But as I have shown here, student learning as measured on achievement tests is only influenced “in part” by the teacher, and that there are many out of school and in-school factors that influence student learning.  We need to stop harassing teachers, step out of the way, and let with them work with their colleagues, and leaderships teams to creative innovate, creative, and healthy learning environments for all students.

Further Reading:

• Problems with the use of student test scores to evaluate teachers. The Economic Policy Institute, August 2010.
• How to fix our schools, Richard Rothstein, Economic Policy Institute, October, 2010.
• Grading our teachers, Maureen Downey, Atlanta Journal-Constitution, January 3, 2011.
• Rating state’s educators, Jaime Sarrio, Atlanta Journal-Constitution, January 2, 2011