Facing the Facts: Why School Policies Backfire

What exactly is policy backfire in K-12 education? To answer that question, we wrote a book. We started by turning this question directly on its head: What is not policy backfire?

First, backfire is not failure. When a school program was intended to produce a specific outcome, and after implementation, the expected outcome was not met — that is failure. Think of the so-called Food Guide Pyramid promulgated by the U.S. Department of Agriculture in the early 1990s or the War on Drugs first declared during the Nixon administration. Neither did what they had intended to do — they didn’t significantly mitigate Americans’ poor eating habits or reduce the public’s use of illicit drugs. They didn’t make things worse, but they didn’t do what they were supposed to either. These programs sort of just fizzled away.

Second, backfire is not unintended consequences. These arise when policies intended for one purpose meet the intended goal but also spur other outcomes for better or for worse. Unintended consequences differ from policy failure because unintended consequences can be good or bad, whereas failure is simply null effects.

Case Studies

So what is policy backfire? Policy backfire is verifiably negative. In fact, backfire leads to negative effects — movement in the exact opposite direction of what was intended. Think of environmental conservation in Kiribati, an island in the Central Pacific. One public policy tried to slow the overfishing on tropical reefs off the coast by paying fishermen to pick coconuts instead of fishing. In theory, this was a good move. But in practice, the fishermen were able to work less to earn the same amount of money as they did as fishing.

So how did these well-compensated fishermen-turned-coconut-farmers spend their free time? Fishing. Paying fishermen to not fish for their wages actually caused them to fish more, and there was an increased problem of coral reef destruction.

And so with this, we turn to education policy backfire. Much of K-12 education policy is devoted to supporting student populations that need additional support, such as non-English speakers or students with disabilities or students who are falling behind in school because of missing support at home. We have seen a swarm of federal, state and district policies and programs designed to reduce gaps in education performance among student groups. And yet, sometimes those policies intending to shrink these gaps have caused the reverse effect — the widening of the achievement gaps. They have, in essence, backfired.

In our book When School Policies Backfire: How Well-Intended Measures Can Harm Our Most Vulnerable Students (Harvard Education Press, 2016), we set out to ask how policies backfire and what we can learn from these situations. We present six cases, dealing with middle school literacy intervention, summer learning loss, minimum grading policies, school closures, school choice and a 1:1 laptop program.

The collection of cases involved researchers from various disciplines, and the examples are drawn from school districts and states across the country. Each used a quantitative or qualitative analytic approach. In collecting these studies, this book provides a critical first look nationwide at the consequences of policy backfire.

Local Context

Two major themes run through the six examples. The first is that backfire occurs as a result of failing to address local circumstance, where the way the policy is integrated within the setting is just as important to success or failure as the policy itself.

In describing a literacy skills intervention in the middle schools of Hampton County (a pseudonym for a suburban East Coast school district with about 90,000 students), Shaun Dougherty, an assistant professor of education policy and leadership at the University of Connecticut, found the program carried some small positive effects for white, Asian and Latino students. But the program backfired for black students, creating an achievement gap that had not previously existed. The negative effects of widening a gap cannot be overstated.

The local circumstances are crucial to discovering the reasons behind this backfire. Dougherty identifies the attempt to set a districtwide qualification for intervention, rather than determining qualification on a school-by-school basis, as a major factor for the failure. 

While using a standardized test score as the districtwide intervention eligibility requirement may have seemed like an unbiased approach, when it was applied to local circumstances at the school level, it became biased. At the school level, when the majority of students below the cutoff are of a particular demographic, their identification and eligibility for the intervention can lead to their feeling marginalized, as was often the case for black students in predominantly white schools. This bias may have long-lasting implications for impacted students.

The second major theme was that backfire occurs due to the failure to provide necessary supports to both staff and students and their families. In New York City, Intermediate School 725 was the type of low-income, low-performing middle school that high school choice policies were intended to help. However, school choice at IS 725 backfired for a reason documented by Carolyn Sattin-Bajaj, assistant professor and co-director of the Center for College Readiness at Seton Hall University.

While the citywide choice policy offered students and parents at the school a vastly expanded set of high school options from which to select, Sattin-Bajaj found school personnel were given little support to manage the pressures and demands of guiding families through this expanded choice system and helping them make informed choices. 

As a result, rather than increasing access for the most disadvantaged students to higher-performing high schools and integrating schools across the city, the program ended up reproducing ongoing patterns and exacerbating school segregation.

Additional Lessons

Beyond these two primary themes, three secondary trends ran through these stories of policy backfire.

First, the researchers discovered the need for strong communication between all players in the schools. This communication must occur across all stakeholders, including but not limited to policymakers, district and school officials, teachers, students and parents.

In his examination of the supplemental literacy course for middle schoolers, Dougherty said the well-intentioned program failed when the logic behind the district-level intervention (how students were identified for the intervention) was not communicated properly to students and their families. As a result, participating students expressed feelings of lower academic self-efficacy.

Similarly, Martha Mac Iver, an education professor at Johns Hopkins University, studied a Texas school district’s top-down decision to use a minimum grading policy. Mac Iver found the failure of district leadership to discuss their intentions and to reach a consensus with opponents led to chaos, acrimony, lawsuits and, ultimately, backfire.

Second, policy backfire results when a strong link does not exist between theory and practice. Closing a failing high school to help students in a large urban district in the South made theoretical sense, but the plans for how to best support the 374 displaced students were not tested first. Hence, students who left the persistently failing school for new settings actually did worse than had they stayed given the lack of supports and resources they were provided at the 23 schools where they enrolled, according to researchers Matthew Gaertner, Ben Kirshner and Kristen Pozzoboni.

A final subtheme regards the role of policy analysis, both quantitative and qualitative. All six case studies exemplify the need to evaluate the effectiveness of policies that we put into place. For instance, when we say that a policy was successful or that it failed, we must ask how do we measure success or failure and for whom?

Andrew McEachin, a policy researcher at RAND, and Allison Atteberry, an assistant professor of research and evaluation at University of Colorado, found the lack of assessment in their study of summer learning loss among students. When statistical models of accountability failed to consider summer learning rates and how they differ for high-need versus low-need students, they contended, comparing spring-to-spring test results hurts those schools serving the most vulnerable students. Hence, when developing analytic models to address the success of new policies, educators ought to consider theory, practice and context to ensure they are arriving at the correct analytic conclusions.

Better Ways

As suggested by the six learning lessons, understanding the reasons for backfire of education policies is critical. Only through studying such cases can we truly understand how and why policies go wrong.

Naturally, we realize it is more appealing to show off success within a school district. It is through studying success, of course, that we can generate a list of best practices. However, only through studying where well-intentioned practices don’t work can we create learning opportunities for decision makers.

As Pedro A. Noguera, distinguished professor of education at the Graduate School of Education and Information Studies at the University of California, Los Angeles, stated in commenting on our study, “[I]f we really seek to improve public education, good intentions are just not good enough.”

Through an understanding of both best practices and backfire we can ensure the success of policies and the school-age children and families they affect.