Feature

Instruments for Evaluation

An array of tools gives depth and breadth to formal assessments of principals by LARRY LASHWAY


During my days as a school board member, the meetings I dreaded most were the ones in which we decided on administrator salaries. Not only did we lack a consistent compensation plan, we had no system for assessing performance. Invariably, meetings would drag on late into the night as board members dredged up random comments they had heard around the community.

 

Our neglect wasn’t unusual. Evaluation of principals often has been an afterthought, consisting of little more than bland checklists or off-handed conversations. In many districts, evaluation is treated more as an irksome bureaucratic requirement than as a vital part of the self-renewal process.

Admittedly, trying to assess the work of principals is a challenging task. Today’s principalship is a complex, demanding, and frequently ambiguous job that doesn’t lend itself to precise analysis. Leadership is like love: trying to dissect it is far less satisfying than simply experiencing it.

But a laissez-faire approach to evaluation shortchanges everyone. As key players in the school community, principals deserve accurate, relevant feedback that not only satisfies the demands of accountability but enhances their performance. One way to get that kind of feedback is to use a formal assessment instrument.

An assessment instrument is any systematic means of generating tangible information about leadership qualities. It may be as simple as a 15-minute paper-and-pencil test or as complex as a hands-on simulation that stretches over two days. While they do not offer a complete solution to the evaluation dilemma, carefully chosen instruments can add depth, breadth and objectivity to principal evaluation, and can promote the kind of self-reflection that fuels professional growth.

Lending Credibility
Assessment instruments put backbone into the evaluation process by spelling out and systematically measuring the criteria for success. A good instrument draws on research or professional consensus to identify key leadership characteristics and then uses those criteria to generate a profile of a principal’s beliefs, skills and style. For example, assessments may reveal how much importance a principal places on team building; how he or she sets priorities; the degree to which the principal is guided by a clear philosophy and vision; and how effectively he or she communicates with teachers.

Of course, supervisors already may know some of these things, and could discuss them in an informal conversation, but a formal assessment does it systematically and objectively, using a large number of specific indicators to build a comprehensive picture. A good instrument compels both principals and their supervisors to look at the big picture, including things they haven’t been aware of (or prefer not to think about).

Formal assessments also bring external credibility to the evaluation process. A principal who is unwilling to confront an issue can rationalize a superintendent’s comments as uninformed or biased but can less easily dismiss the results of an objectively scored test that reflects the professional consensus of experts. Conversely, assessments can boost confidence and self-assurance by uncovering latent strengths that have previously gone unrecognized. Even principals who are already aware of their strengths are pleased to have them confirmed by a credible assessment process.

Many assessment instruments further enhance their accuracy by providing feedback from superiors, subordinates and peers. This 360-degree feedback helps create a multifaceted portrait by cross-checking the principal’s perceptions with those of the people who know him or her best.

Assessment instruments provide concrete data that can be analyzed, pondered and used as a spark for professional development. Consider a principal who frequently endorses the value of collaboration and teamwork and sees himself as an empowering leader. Yet the assessment instrument shows attitudes that usually are inconsistent with collaboration, and the feedback from teachers reveals that whatever the outward signs of teamwork, they don’t feel empowered. At the very least, the discrepancy would lead to further questions about how this principal is actually operating. Is his commitment to empowerment simply rhetorical? Does he define empowerment differently than the teachers? Does the school lack the right structures for collaborative leadership? In fact, the biggest value of formal assessments is not the answers they provide, but the questions they generate.

Proper Application
Despite the insights that assessment instruments can provide, they aren’t panaceas. They can tell you how principals perceive themselves (or how others perceive them), and they can provide a snapshot of leadership skills. But because they do not measure results--what the principal has achieved in a real-world setting--they are poor tools for accountability, and most test publishers discourage their use for this purpose.

High-stakes decisions such as promotion, retention and compensation require attention to the bottom line. Are students in this school learning? Are discipline problems diminishing? Is faculty morale high?

However, assessment instruments can play a crucial role in the overall evaluation process. The weak link in most performance evaluations is determining cause and effect. Test scores are up? Fine, but is it the result of the principal’s actions or just a minor demographic shift? Faculty morale has improved? Excellent, but exactly how has the principal achieved this? Because assessment instruments are based on extensive research or professional consensus on best practices, they can point out which behaviors most likely made the difference. Establishing a link between action and outcome creates the foundation for professional improvement.

Choosing an Instrument
Every assessment instrument offers its own take on leadership. Some are concerned with style, some emphasize skills and others focus on beliefs. Some lean toward routine management skills, while others highlight vision and inspirational leadership. Some can be quickly administered and self-scored, while others require a day or two off-site. Choosing the right one for your district takes some effort, but is worth the investment.

The following guidelines outline the essential steps in the process.

  • No. 1: Decide what you want to know.

    The first step is the hardest: deciding what leadership behaviors, perceptions and skills are most important for principals in your district. It’s an intimidating task, particularly if your district hasn’t given much conscious thought to the question.

    But it’s a critical decision, since effective leadership is often situational. What works in one place may flop in another. Everyone knows principals who failed simply because their style or skills didn’t fit the school; often these same individuals are highly effective in their next assignment. Importing a generic assessment won’t do justice to your schools, which aren’t generic.

    Thus, you need to determine what particular leadership qualities are critical for success in your district. A good starting point is to look at the recently developed standards from the Interstate School Leaders Licensure Consortium. An umbrella group with membership from 24 states and professional associations, including AASA, ISLLC has boiled down its definition of leadership into just six standards, ranging from facilitating vision to nurturing school culture.

    The standards include specific success indicators. For example, the standard for vision includes indicators such as "the vision is developed with and among stakeholders," "assessment data related to student learning are used to develop the school vision and goals" and "existing resources are used in support of the school vision and goals."

    The ISLLC standards provide a good foundation for thinking about assessment, but the list should be refined and adapted to fit the leadership priorities in your district. Principals themselves can tell you a lot about the issues they’re currently struggling with and what kind of feedback would be most useful. Other constituencies, such as teachers and parents, also can provide helpful perspectives on assessment needs.

    Ideally, this analysis will recognize differences at the school level. A school with a confident veteran staff requires different leadership skills than a demoralized school with high turnover. However it is done, identifying assessment priorities provides the critical foundation for selecting the right instrument.

  • Testing Relevance
    No. 2: Locate the instruments most appropriate for your purposes.

    At one time, leadership instruments were difficult to locate and examine, but several sources now offer centralized listings, either published or on line (see related story). Most publishers will provide examination copies or detailed descriptions of their tests.

    In reviewing assessments, it’s important to remember that no instrument can be all things to all users. Each test takes some stance on leadership that may or may not be a good match for the leadership priorities in your district. If you’ve identified "building school culture" as a leadership priority in your district, then the test you choose should provide solid feedback on that dimension.

  • No. 3: Judge the credibility of the test.

    Nothing undermines assessment faster than the perception that the process is inaccurate, irrelevant or unfair. Solid evidence should demonstrate that the instrument is statistically sound and capable of providing the information desired.

    Most test publishers will provide information on reliability and validity, some of which can be difficult for non-statisticians to evaluate, but anyone can ask two common-sense questions that will get to the heart of things:

  • Nothing undermines assessment faster than the perception that the process is inaccurate, irrelevant or unfair. Solid evidence should demonstrate that the instrument is statistically sound and capable of providing the information desired.Most test publishers will provide information on reliability and validity, some of which can be difficult for non-statisticians to evaluate, but anyone can ask two common-sense questions that will get to the heart of things:
  • Does the instrument assess the qualities it claims to? Would a reasonable person agree that the items in this assessment reflect the particular dimensions of leadership that we’re interested in? If the answer is yes, the assessment will be seen as credible.

  • Is there evidence that performance on the assessment correlates to performance on the job? That is, do high scorers perform better on the job than low scorers? Not every instrument has been validated in this way, and even when the information is provided it has to be interpreted carefully. (For example, many tests were developed in a corporate environment and the results are not automatically transferable to school settings.) Over time, however, you should be able to determine that a particular instrument is a good predictor of principal success in your district.

    Credibility also is improved when the assessment includes multiple perspectives. A growing number of tests provide 360-degree feedback, in which the leader’s self-assessment is supplemented by feedback from supervisors, teachers and peers. Each group sees leadership from its own limited and possibly biased viewpoint, but using all perspectives permits a reasonable triangulation of the leader’s actual performance.

  • Followup Assistance
  • No. 4: Determine how much support the instrument provides for followup and professional development.

    Most formal assessments come with post-test activities that help leaders reflect on the results and set goals for continuing development, but they vary in quality. The key questions: Are the test results reported in a clear, understandable format? Do the supporting materials provide test-takers with insights into the meaning of their leadership profile? Does the test package help administrators set an agenda for improvement?

  • No. 5: Consider the practical issues.

    How long does the test take? Can it be scored locally? What’s the cost? No instrument will satisfy all your purposes, and the final choice will be the result of numerous trade-offs.

  • No. 6: Once the assessment has been administered, provide support for professional development.

    The best instrument in the world will do little good if the results are simply recorded and filed. Results should form an agenda for continuing reflection and professional development activities (see related story). The district’s willingness to provide tangible support for this activity sends a clear message that assessment is a valued process that will be taken seriously.

    As much as we might dream of a simple off-the-shelf test that would painlessly solve our evaluation problems, effective assessment requires careful thought and nurturing. But the payoff for that investment can be significant. Research tells us that reflection plays a critical role in education. Principals, like teachers, learn from experience but only if they assess that experience in a thoughtful, reflective manner. The right assessment instrument can provide a powerful tool for that kind of reflection.

    Larry Lashway is coordinator of the collaborative teacher education program at Grays Harbor College, 1620 Edward P. Smith Drive, Aberbeen, Wash. 98520. E-mail: llashway@ghc.ctc.edu. He also is research analyst with the ERIC Clearinghouse on Educational Management.
  •