Feature Pages 27-31
Charlotte Danielson on Teacher
Evaluation and Quality
A School Administrator interview with the creator of the Framework for Teaching
BY LIZ GRIFFIN
Charlotte Danielson is an expert on good teaching, having analyzed the performance of what distinguished, proficient and underperforming teachers do (or don’t do). Her detailed rubrics on teaching reveal the benefits to teachers, students and administrators when a common standard of practice is shared.
Policymakers and educators across the country have embraced her work as they improve their teacher evaluation systems. Many use her framework as a foundation.
In a recent interview with School Administrator, Danielson talked about what needs fixing in teacher evaluation, how superintendents can support teachers’ and principals’ work, and what components make up a strong evaluation system. She will be a General Session presenter at AASA’s national conference in February.
How has your work evolved over the past 15 years?
DANIELSON: When I published the Framework for Teaching in 1996, it provided the first detailed description of good teaching. It retains the same architecture, the same 22 components, but I’ve added clarity by creating critical attributes for different levels of performance and providing examples from classroom practice.
From the beginning, the framework was an instrument for teacher evaluation. It allows observers, evaluators and supervisors to use precise language to distinguish teacher performance from one level to the next. And it allows teachers to understand the criteria on which they will be evaluated and to reflect on practice.
The tool grew out of my experience with high-stakes assessment projects, including the National Board for Professional Teaching Standards and a project at Educational Testing Service called Praxis III, which was an observation-based assessment used for teacher licensing. What they had in common were clear standards of practice.
Districts use different models to evaluate teacher effectiveness. Do you think growth models in use today evaluate individual teacher effectiveness fairly and accurately?
DANIELSON: The short answer is no.
Essentially, there are two approaches to assessing teacher effectiveness. One is the work of teachers, that is, how well they do the work of teaching. That’s what my framework permits.
The second major approach is, as you suggest, based on the results teachers get with students. How well do their students learn? Ultimately, that is the bottom line, of course. It’s why we have schools.
The challenge is there must be measures of student learning that, in fact, are valid, reliable and fair to teachers. One truth is many factors contribute to student learning, in addition to the quality of teaching in a given year. Those things must be taken into account if it’s going to be fair to teachers.
Two questions in this debate are “What do you count as evidence of student learning?” and “How can you fairly attribute it to the work of individual teachers?”
In my view, nobody on the planet has figured this out because it’s extremely difficult to do.
MORE ON DANIELSON:
Charlotte Danielson will be a Thought Leader presenter on the first day of AASA’s national conference in Los Angeles in February.
The Danielson Group, based in Princeton, N.J., has additional information about her teacher evaluation frameworks on its website (www.danielsongroup.org).
Can you explain the difficulties?
DANIELSON: Let’s say I teach 4th grade and my students’ scores on pre-post assessments in reading have increased a lot. I’m happy, my principal’s happy, and the parents are happy.
But it’s hard to know that I was the one who actually caused that gain. It could well be there’s a reading specialist in the building. Or it might be last year’s teacher had some great strategies and students are still using them.
It might not have much to do with me. So until somebody has figured that out, we’d better not be making high-stakes decisions about teachers’ performance.
There also are psychometric and measurement challenges that one confronts when addressing teacher practices. They have mostly to do with the training of evaluators. The good news is we now know how to do that, as a result of our research. That’s not the case with the measures of student learning.
Why is your framework more effective than other models?
DANIELSON: My framework has been used and refined over 15 years. The language has tightened. Another thing is that my framework for teaching has been used in several very large, independent research studies. One was done by the Consortium on Chicago School Research. The other was Measures of Effective Teaching [the MET study], funded by the Gates Foundation.
In the MET study, something like 23,000 lessons were captured on video and then analyzed according to five observation protocols, of which mine was one. The research found my framework had predictive validity. That means when teachers perform well [on the framework] as judged by trained and certified assessors, their students perform better and learn more than the students in the classrooms of teachers who don’t perform so well.
I don’t think other models have been subjected to that level of scrutiny.
Your framework identifies 22 essential attributes of good teaching. Should a teacher focus on one component as more important?
DANIELSON: We don’t yet have data that show which components of the framework are most highly correlated with student learning. The heart of my framework is student engagement and everything that supports that process, but other things are important, as well. For instance, classroom organization and manageable routines and procedures are not trivial matters, especially for beginning and first-year teachers. They make student engagement possible.
I don’t think you can do these things in isolation.
What can school district leaders take away from the results of the MET study?
DANIELSON: They need an evaluation instrument that is valid, rigorous and reliable. The second thing is it’s essential people be trained to a sufficient level to make accurate, consistent judgments about a teacher’s performance. We know how to do this. For the MET study, we trained observers, entirely online, and they took a test. They passed at astonishing rates, about 93 percent.
But they found they need that training. That’s because the thinking you have to engage in when you’re being specific about whether a teacher is performing at a Level 2 [basic] or a Level 3 [proficient] is different from what most people have ever had to do.
What lessons have district leaders learned from past implementation efforts?
DANIELSON: When my framework was first published, a large urban district wanted to revise its evaluation system. It had been a terrible system — top down, punitive and arbitrary. They revised it based on my book and, you know what, it was top down and punitive! All they did was exchange one set of evaluative criteria with another. They did nothing to change the culture of teacher evaluation. The teachers were understandably very upset.
It’s more likely teacher evaluation reform will succeed when efforts start small and build over time. Ideally, districts could take a year and talk about the framework as a tool to improve instruction before using it as an evaluation tool. Districts and state efforts that are successful include pilot projects to allow people sufficient time to gain a deep understanding of the framework and what good teaching looks like.
Changes of this scope are complex. How do you change the culture and capacity of a district?
DANIELSON: It’s an instructional leadership issue and a professional development issue. It involves professional learning and conversations with site administrators and teachers. The more conversation the better.
Site administrators as instructional leaders must appreciate the role of school culture, a professional culture, a culture of professional inquiry. They must define teaching as not just what you do with your kids for six hours a day but also about building a professional culture in which everybody is still learning.
One central theme of your work is that the teacher evaluation can serve as an opportunity for professional development.
DANIELSON: That’s true. Evaluation of teacher practice has to identify teachers who are truly performing below standard and need to be coached or maybe even dismissed. That has to be part of what any evaluation system is capable of doing. But we have to be able to create educative systems that actually result in learning, that are worth doing from the standpoint of teachers.
In my framework, the principal and teacher are engaged in conversation. They compare notes on what happened in the class and interpret it against a rubric that provides clear attributes of what teachers do. For example, they look at what happened in the classsroom around questioning and discussion and how that differs at four levels — unsatisfactory, basic, proficient or distinguished.
We know this model vastly improves the professional conversation around teacher practice.
It’s about training and training and training. The unions understand this well. And, in some places, they’re actually bargaining for it.
Do you have advice for superintendents?
DANIELSON: It’s the superintendent who sends the signals. The message for superintendents has to be to make the supervision of instruction and the improvement of teaching the most important priorities of site administrators.
Superintendents are the ones who help principals organize their time. They must work out with principals how to, if not reassign some duties they do now, at least re-establish principals’ priorities.
It might be a matter of enlisting people to do the jobs in the school that require management, but not instructional leadership. There’s no doubt building management has to happen. But it may not take someone with a Ph.D. to do it.
So, instructional leadership is a personnel challenge for superintendents, and one of resource allocation.
How do superintendents support principals and teachers? You’ve talked about the need for professional development and professional conversations.
DANIELSON: My experience is many principals still don’t understand what instructional leadership means or how to do it. Many teachers and, indeed, some administrators lack understanding of what student engagement is. They think so long as the kids are busy, they are engaged. Yet it’s a more evolved concept than that. Student engagement looks at the nature of kids’ thinking.
So these are PD issues. Superintendents must help teachers and principals find time to collaborate. And not only to have the time to talk, but the skills to have professional conversations. Most administrator preparation programs don’t do a good job at teaching these skills. In many places, that’s the missing piece.
Superintendents are the ones who can ensure evaluation system procedures invite teachers to engage in activities that we know promote professional learning. Those things include teachers’ self-assessment, reflection on practice and engaging in professional conversation. These processes should be embedded in the evaluation system.
How often should teachers be observed?
DANIELSON: Frequent, unannounced and brief observations let you keep your finger on the pulse of teaching in your school. It’s actually better if some observations are not part of the formal evaluation process because you develop a professional community where everyone uses the same definition of teaching, not just a system for personnel decisions. So the more observations, the better. And the more conversation, the better.
If you can manage three short observations, that’s a lot better than one longer one. Peers, department heads, mentors or coaches can serve as observers. When schools do this, it builds the professional conversation about practice. You all become more effective because you know the challenges you face.
For evaluative purposes, you want a sampling that represents typical teaching, which the MET study found required at least three — and better, four — observations by a trained and certified assessor to be reliable. Now, most schools can’t manage that. I mean, that’s a real burden on personnel. Time is a limiting factor.
Let’s focus on a related topic: classroom observation. What does rigorous training for classroom observers entail?
DANIELSON: In our training we show (people) multiple examples of teaching practice, multiple videos for all 22 components of my framework and examples of each performance level. We have examples for, say, a proficient level of questioning and discussion skills, and a video reflecting high proficient but not distinguished, or low proficient but not basic. Then participants review written rationales and annotations for why an example was rated at that level.
We have something over 100 videos. That’s a tremendous resource for administrators, certainly, and, indeed, for teachers, to understand what practice looks like at the general levels.
How much time does it take to do a quality classroom observation?
DANIELSON: One surprising finding of the MET study is that if you do shorter observations you get as accurate a picture of what’s going on in a classroom, in fact slightly more accurate than one 45-minute observation period. It’s also a bit of a surprise that the MET study found it didn’t matter which 18 minutes you observe: the beginning, middle or end of a lesson.
These findings are good news for evaluators and observers. Obviously, shorter, frequent observations permit observers to see a much bigger sample of teaching. An observer may view a math lesson, a social studies lesson and a reading lesson at the elementary level. Or, at the secondary level, different class periods within the day. Shorter observations also make scheduling them more manageable.
While many states and many bargaining agreements mandate that for evaluation purposes the observer must stay for a full period, the encouraging thing is that observers don’t have to stay for an entire class period for observations to be statistically valid.
How brief? What does an observer look for during a classroom visit?
DANIELSON: Five minutes and 10 minutes is long enough time to sense the nature of learning that’s going on.
If we believe learning is done by the learner through this active intellectual process, then we have to care about not only what the teacher is doing, but also what the students are doing. What is the nature of their tasks? Of the learning? Do they have to think? Do they have to use their minds?
Administrators face an enormous paperwork burden. How can they cope?
DANIELSON: I’ve partnered with a software company to design online tools that help administrators. They’ve developed an iPad app that helps with assessment of observers. Many vendors sense there is money to be made and are getting into the act. I’ve partnered with just one company so that I could ensure a high quality product.
In response to demand, the Danielson Group also developed a commercial version of the online training and certification test for administrators and observers that we used in the MET study.
You’ve acknowledged that time is a limiting factor. What did you mean?
DANIELSON: State policymakers have set aggressive timelines for implementation of new teacher evaluation systems.
Administrators are simultaneously working on aligning curriculum and assessment with the Common Core. These are challenging, time-consuming tasks.
It’s understandable that administrators can feel overwhelmed. The pressure to move quickly to employ the framework as an evaluation tool is very real.
However, moving too quickly makes everyone nervous and can be self-defeating.
What will be the Common Core’s impact on teacher evaluation?
DANIELSON: If new assessments are true to what the standards say, they are going to be extremely rigorous. The Common Core standards represent extremely high-level learning, much higher than what we’ve had in this country for all but the most elite and best-prepared students. I frankly welcome that.
But those are going to set other things in motion. If you have very rigorous assessments and a huge failure rate, that presents a challenge for a lot of teachers. So the rollout of all these changes will have at least as big an impact as anything I’ve been doing.
Imagine that you hopped in a time machine and traveled 10 years into the future. What impact do you think you would find from the new teacher evaluation methods?
DANIELSON: That’s an interesting question. Well, one possibility is that, 10 years from now, nothing will have changed. People will be onto the next “best” thing.
On the other hand, it’s a healthy development to be thinking hard about teacher quality and considering what it means and how you strengthen it.
I would love to come back in 10 years, from my warm beach, and discover that teaching is better than it was. And find that teachers are more thoughtful, and administrators have time to talk with their staff about what they’re doing and what the kids are learning.
That would be delicious.
Liz Griffin is managing editor of School Administrator. E-mail: email@example.com