Over the past three years, the Bill & Melinda Gates Foundation has lead some of the most influential work around multiple measures in education through their Measures of Effective Teaching (MET) project, a partnership of more than 3,000 public school teachers who voluntarily opened up their classrooms to researchers. The study looked at three measures: value-added analysis, evaluation, and student surveys with the purpose of investigating “better ways to identify and develop effective teaching” as well as “help teachers and school systems close the gap between their expectations for effective teaching and what is actually happening in classrooms.” Participating districts included Denver Public Schools, Dallas Independent School District, Memphis Public Schools, Pittsburgh Public Schools, New York City Schools, Charlotte-Mecklenburg Schools, and Hillsborough County Public Schools.
In January 2013, the MET project released their final report, “Ensuring Fair and Reliable Measures of Effective Teaching: Culminating Findings from the MET Project’s Three-Year Study.” The report presented several key findings which impact teachers, building-level leaders, district and other administrators, as well human resources staff and processes, including:
• Great teaching CAN be measured.
• Teachers need meaningful feedback to grow.
• Observations should be done by multiple reviewers, multiple times. Specifically, the report notes that shorter, more frequent observations from two or more observers per teacher provide a more reliable snap-shot of true teacher performance, rather than one individual performing a single, longer observation.
• Building processes that increase trust and fairness will result in better data.
• Surveying students? Ensure confidentiality. The report notes that student survey data becomes more reliable when students feel that they are able to provide anonymous feedback.
• Utilize multiple measures when building teacher evaluation or performance index formulas. The report states, “Compared with schemes that heavily weight one measure, those that assign 33 percent to 50 percent of the weight to student achievement gains achieve more consistency, avoid the risk of encouraging too narrow a focus on any one aspect of teaching, and can support a broader range of learning objectives than measured by a single test.”
Randi Weingarten, president of the American Federation of Teachers, said in a statement, “The MET findings reinforce the importance of evaluating teachers based on a balance of multiple measures of teaching effectiveness, in contrast to the limitations of focusing on student test scores, value-added scores, or any other single measure.”
In recent years, several states and districts across the country have made changes to their educator evaluation policies, while many others are currently considering reforms. Applying lessons from the MET project and similar work, there are some best practices that all HR directors and other education leaders should consider around teacher evaluation.
1. Communication is essential. Ensure you have a process that is documented, communicated, and available online for everyone to see. Even if the state department of education or state legislature has made the policy, make sure you understand it, know where to go with questions, and are communicating the status of the work with staff regularly. Communication with parents and stakeholders is also important.
2. Select great measures. Unless your data is coming from a state contract, work carefully to select a vendor who supplies the information, level of customer service, and transparency you need. Commissioned by the Bill & Melinda Gates Foundation, Battelle for Kids developed a free website, www.edgrowthmeasures.org, with information to consider when selecting a growth measures provider.
3. Train, check, train, recheck, and train again. Those responsible for performing classroom observations must be trained and evaluated continuously, particularly when teachers’ evaluation scores are being tied to high-stakes decisions around compensation, tenure, promotion, or dismissal. It is also important to keep the training process transparent.
4. Work constantly to build understanding. Ensure educators know how their performance is being measured and evaluated. Find someone who can not only help staff understand the data, but how best to use it to improve their practice.
5. Encourage questions and feedback. There are many places in the evaluation process where error or issue can occur. Giving those evaluated the ability to identify the issue and voice a resolution is important for fairness, validity, rigor, and legality.
6. Be strategic in weighting measures. The MET report found that evaluation systems which “assign 33 percent to 50 percent of the weight to student achievement gains achieve more consistency… and can support a broader range of learning objectives than measured by a single test.” Multiple measures–weighted appropriately–are critical to developing an evaluation system that is reliable and helps educators improve their practice. It is also important to remember that it’s not necessarily the more measures the better, but the more RIGHT measures the better.
7. Keep an eye on your data: If preliminary evaluation/observation data doesn’t match the other performance data you are collecting, look for ways to ensure validity of evaluation scores. For example, the MET project found that evaluations of teachers done by multiple individuals and someone other than their principals may yield more accurate results. Don’t get to the end of the school year and then realize that there are issues with the data that could have been addressed early on.
8. Practice comprehensive and strategic human capital management: Align your evaluation system to other human capital initiatives, such as hiring, professional development, and succession planning. It is essential to give teachers and principals the chance to reflect on their practice as well as time, resources, and opportunities to improve.
Teacher evaluation will continue to be a hot topic in states and districts across the country. No matter how your district chooses to measure and evaluate educator effectiveness, the MET report and similar research from across the country offers important lessons to consider in designing and implementing these systems. It will be interesting to see how this work influences federal, state, and local education policy in the years to come.
This article first appeared in the May/June/July issue of the American Association of School Personnel Administrators, Best Practices in School Personnel: Teacher Evaluation & Multiple Measures, magazine. Used with permission from the author and AASPA.
“Ensuring Fair and Reliable Measures of Effective Teaching.” Measures of Effective Teaching Project. The Bill and Melinda Gates Foundation. January 2013. http://metproject.org/downloads/MET_Ensuring_Fair_and_Reliable_Measures_Practitioner_Brief.pdf
“Feedback for Better Teaching: Nine Principles for Using Measures of Effective Teaching.” Measures of Effective Teaching Project. The Bill and Melinda Gates Foundation. January 2013. http://metproject.org/downloads/MET_Feedback%20for%20Better%20Teaching_Principles%20Paper.pdf
“Gathering Feedback for Teaching: Combining High-Quality Observations with Student Survey and Achievement Gains.” Measures of Effective Teaching Project. The Bill and Melinda Gates Foundation. January 2012. http://www.metproject.org/downloads/MET_Gathering_Feedback_Practioner_Brief.pdf
Ho, Andrew D. and Thomas J. Kane. “The Reliability of Classroom Observations by School Personnel.” Measures of Effective Teaching Project. The Bill and Melinda Gates Foundation. January 2013. http://www.metproject.org/downloads/MET_Reliability%20of%20Classroom%20Observations_Research%20Paper.pdf
Kane, Thomas J., Daniel F. McCaffrey, Trey Miller, and Douglas O. Staiger. “Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment.” Measures of Effective Teaching Project. The Bill and Melinda Gates Foundation. January 2013. http://www.metproject.org/downloads/MET_Validating_Using_Random_Assignment_Research_Paper.pdf
On Education Week K-12 Talent Manager: