Teacher Ratings in a PLC World

Barry J. Vitcov

Introduction

Vitcov-150px.jpg 
 Barry J. Vitcov
Regardless of how well a new leader is prepared for the job, there are always unanticipated challenges. As a leader moves through new roles towards a superintendent’s job, it is customary to begin work as listener learning about the organization’s culture while developing an improvement agenda. Some challenges may require superficial changes, such as position reassignments or new structures for administrative meetings; others may require deeper systemic changes, such as governance and communication structures, processes and expectations. Whatever the challenges, they will all require a new superintendent to shape a district culture that is effectively proactive and reactive to whatever comes its way. One challenge, which will always be a high priority, is how to maximize continuous improvement of teaching and learning. A number of stakeholders will influence how to address this challenge. Some examples of initiatives intent of improving teaching and learning are: implementation of common core standards and their new assessments, proficiency grading and improved supervision and evaluation systems. No doubt each of these initiatives requires lots of attention. All of these initiatives also require systems that support coherent practice among administrators and teachers. Lots of attention has been given to improving supervision and evaluation of teachers, highlighted by the need to get teacher evaluations “right”.

The need for improved teacher evaluations has increased the pressure to insure inter-rater reliability among those responsible for supervising and evaluating teachers. While supporting principals to skillfully and accurately calibrate teacher ratings is important and necessary, I believe there are greater opportunities for supervision and evaluation processes that take a systems approach, which can lead to both inter-rater reliability among school administrators and intra-rater reliability between administrators and teachers. In my work with large and small school districts, the most effective supervision and evaluation processes have been built as a district system that actively involves central office and school administrators along with full teacher participation. When everyone in the system participates in the calibration of teacher practice, evaluations have the potential to be more meaningful, reliable and valid. Too many, if not most, school districts are driven to have their administrators “get evaluations right”. This intense focus on “getting it right” in terms of principals evaluating teachers limits the opportunity for using supervision and evaluation to grow a collaborative learning community that honors everyone’s professional expertise along with creating targeted and vibrant professional development for both administrators and teachers.

System-wide participation requires everyone to learn and understand a common instructional language rooted in the data collected around teaching and academic content standards. There are three critical questions that need to be answered if supervision and evaluation processes support a system-wide professional learning culture: First, how do the processes support deeper and shared understanding of teaching and learning? Second, how are the processes transparent and ones in which the principal and teachers have a sense of joint ownership? And, finally, how do the processes impact student learning?

How do the processes support deeper and shared understanding of teaching and learning?

Most educators agree that teacher supervision and evaluation ought to serve two basic purposes. The first is to insure a standard of teacher quality by which employment decisions are made. Clearly, the profession has not done a good enough job of retaining high quality teachers - especially in their first years on the job – nor has it done an adequate job of ridding the profession of those teachers not meeting standards. The second purpose is to continuously support teacher growth and expertise throughout their entire career. When I ask teachers what it means when a principal is in their classrooms, whether it be a short or more extended appearance, invariably the response is, “My principal is evaluating me.” When I ask principals what they believe teachers are thinking when they are in their classrooms, invariably the principals’ response is, “They believe I am evaluating them.” An emphasis on a principal getting ratings right emphasizes judgmental behaviors and diminishes the opportunity to support professional growth. This is exacerbated by the common practice of training administrators and teachers in isolation from one another. For example, it is common for districts to support inter-rater reliability among principals by having them view classroom video clips and then assess teacher practice on an adopted rubric of teacher standards. Unfortunately, this kind of professional development for principals is infrequent and it doesn’t include teachers who will be subject to their principal’s judgment. Teachers will always be suspect of supervision and evaluation if they believe the process is something that is done to them. Ongoing professional development where principals and teachers reflect on instruction and student outcomes as part of a joint learning process helps to shift what is often perceived as a negative process to a positive one. This ongoing process allows for a threaded story to develop over time about teaching and learning, which will result in more genuine evaluations. One simple solution to principal-only inter-rater reliability training is to have principals and teachers view video clips and calibrate teacher practices together.

There is an assumption that calibrated observations of teacher practice can be measured with a reliable degree of exactitude to a scaled rubric of teaching standards. The problem with that assumption is that the nature of all scaled rubrics is inexact. Rubric language lacks finite clarity and requires interpretation and deeper understanding of what is written. When the addition of varied classroom contexts and observer bias is taken into consideration, ratings become even more skewed. Principals and teachers collaboratively viewing classroom video clips, or, even better yet, conducting classroom walkthroughs, provides an opportunity to create common understandings of what teaching standards and rubrics mean. For example, most descriptions of effective teacher practice will include some language about “engaging students” without any specific examples of what that looks like. Does it mean students are talking to one another in cooperative learning groups? Does it mean students are working on cognitively demanding tasks? These are the sorts of inquiries principals and teachers ought to talk about together. When it’s only a principal assessing teacher practice, it’s all judgmental and evaluative. When principals and teachers engage in conversation about a classroom observation, there is the potential, especially when the principal uses language that elicits teacher reflection, for collaborative learning. A disturbing trend I’ve seen with supervision and evaluation is the use of electronic devices, popularized by iPad applications, on which principals record a classroom observation that is subsequently emailed to the teacher. While this practice is accepted by many as a boon to efficiency and thus more frequent classroom observations – frequent classroom observation being a good practice – it reduces dialog between principals and teachers and emphasizes the principal as judge and jury. The system needs to support principals to do frequent classroom observations with frequent conversations with teachers about teaching and learning.

One big problem with calibrating to a rubric is the misunderstanding of what is meant by “best practices” and “research-based” instruction. Robert Marzano has helped us to understand how the term “best practice” has been a misinterpretation of his concept of “high-yield strategies.” Teacher rubrics suggest “best practices” without distinguishing contexts. Far too many principals are using lists of “best practices” to check-off what they observe in classrooms. This check-off mentality is an attempt to use the science of teaching without honoring the art of teaching. And one rater’s bias towards one particular instructional practice can strongly influence how they use a rubric. It is through active dialog with and among principals and teachers that shared understandings of effective practices will ultimately impact student achievement.

How are the processes transparent and ones in which the principals and teachers have a sense of joint ownership?

Imagine a school district’s central office administration meeting with their principals and asking them to actively engage their teaching staffs in a conversation and identification of specific instructional strategies, perhaps tied to a recent professional development session, which will be used as a focus for upcoming classroom observations. Furthermore, the data from those observations would be shared and analyzed by principals with each other and with their teaching staffs. This type of system’s approach, embedded in a professional learning community in which classroom observation data is public and transparent, has the potential to positively impact teacher practice and student learning much more than a system where administrators own the practice.

It is rare when principals ask teachers to describe what ought to be observed in an effective classroom. However, I have visited schools where the focus of classroom instruction is identified by whole school, grade-levels, or subject-area departments for subsequent classroom observations. Sometimes those observations are made by an administrator, small groups of teachers, or administrators and teachers together. Often times, it’s the principal who asks teachers questions such, “What should I look for when you are teaching a lesson that is applying knowledge from prior instruction?” The teachers might identify signals for bringing forth prior knowledge, or the kind and type of questions they might be asking, or, better yet, the type and quality of student responses they are expecting from their instruction. The principal might clarify teacher responses and even add to the list. It is through these kinds of discussions that supervision becomes a shared process, and those shared processes can result in more accurate and community-calibrated evaluations. It allows for a sense of intra-rater reliability between administrators and teachers. When the central office involves principals in frequently using this sort of community-generated knowledge with one another, it builds inter-rater reliability among principals.

How do the processes impact student learning?

No doubt good teaching leads to improved student achievement. Teacher supervision and evaluation must pay greater attention to the impact teaching has on student learning. The problem with relying only on standards of teacher practice to assess quality is that those standards rarely take into consideration student achievement. The standards focus on teacher behaviors. When we ask principals to calibrate their judgments only using standards-based rubrics, we are ignoring what is most important: student achievement. I’ve conducted hundreds of informal classroom walkthrough observations with principals, and, without exception, when I ask principals beforehand to tell me to what they pay attention during observations, they say, “How well teachers are meeting teaching standards.” In a nutshell, very little, if any, attention is paid to what students are doing and to what degree they are being academically successful. What I ask principals to do, instead of simply watching what the teacher is doing, is to first look at what the students are doing.

We’ve known for a long time that the alignment of content (what is taught), instruction (how content is taught), and assessment of student achievement is fundamental to high functioning schools. Supervision and evaluation must pay heed to more than assessing instruction on teaching standards’ rubrics. Looking at what students are doing before strict attention to what teachers are doing encourages principals and teachers to examine how student achievement was a result of instruction. Principal and teacher dialog that connects student outcomes with teacher behaviors is an important step in determining next learning targets, instructional strategies, and assessment of student learning.

The push for principals to get their teacher ratings right might go in two directions. One will be a one-way street where principals simply tell teachers how they are doing. A principal’s confidence in this system is built around their inter-rater reliability with their fellow principals. If the desired outcome of inter-rater reliability is compliance to a set of standards, then that’s what we’ll get. If, however, the desired outcome is a system that is both built on compliance to a set of standards as well as support for achieving those standards, then a more powerful system will be a two-way street. In this scenario principals and teachers will both be involved in developing inter- and intra-relater reliability that results in a healthier, more dynamic learning community. It is my hope that new superintendents will shape a district culture in which the system is in charge of teacher quality and not just a handful of administrators. It is my hope that new superintendents will shape professional learning cultures that sustain continuous improvement of teaching and learning.
--------------------------------------------------------------------------------

Barry J Vitcov
Senior Consultant
New Teacher Center
430 Briscoe Place
Ashland, Oregon
Barry.vitcov@gmail.com