Accountability for Noncognitive Skills

Society values traits not covered on academic tests, so why aren’t they measured in school? by Richard Rothstein
Federal and state education policy relies on the belief that student achievement will improve if schools have incentives and disincentives — rewards if they succeed, sanctions if they fail — to raise test scores in math, reading and science.

But even if this were a practical way to elevate such scores, a test-based accountability system is still harmful because incentives, if they work, direct institutions away from activities that are not rewarded and toward activities that are. Once incentives are attached to standardized test scores and not attached to other outcomes, testing must inevitably result in less attention paid to less-tested parts of the academic curriculum (such as the arts, social studies and physical education) and in less attention to the noncognitive goals of schooling — character traits like perseverance, self-confidence, self-discipline, punctuality, communication skills, social responsibility and the ability to work with others and resolve conflicts. These noncognitive outcomes are important goals of public education, perhaps even more important than the academic ones.

After all, little disagreement exists that Americans want outcomes besides literacy and numeracy from schools. While public opinion surveys consistently include higher test scores as a school goal, the public does not believe that test scores are the most crucial goal. In one recent survey, more than two-thirds of Americans said teaching values was a more important public school role than teaching academic subjects. The top-rated value was teaching students to solve problems without violence. This judgment only restates what Thomas Jefferson called upon public schools to accomplish — not only raise students’ academic proficiency but ensure they will "understand duties to neighbors and country, and ... observe with intelligence and faithfulness all social relations ... ."

In lawsuits in the various states over the past three decades, plaintiffs have demanded increased state funding to guarantee adequate educations. These claims require a definition of "adequacy" before being able to specify how much money schools need to generate it.

Most courts have proclaimed broad definitions. In 1979, the West Virginia Supreme Court provided a model that has been subsequently followed by other states. The court ordered the West Virginia legislature to provide sufficient school funds to ensure that all children have, in addition to literacy, numeracy and civic knowledge, the self-knowledge to "choose life work and to know their options," the capacity as adults for "recreational pursuits, interests in all creative arts, such as music, theater, literature and the visual arts," and a sense of "social ethics … to facilitate compatibility with others."

Surveys conducted by business groups like the National Association of Manufacturers, by government agencies like the Census Bureau and Labor Department and by education policy groups consistently have shown that employers complain far more about job applicants’ communication skills, punctuality, responsibility, attitude, teamwork ability and conflict resolution skills than about their verbal and mathematical levels. Employers report so reliably their concern with noncognitive skills, it is remarkable that schools are held accountable mainly for better test scores, purportedly because these are needed to prepare graduates for the workforce. Curiously, business lobbyists support laws that hold schools accountable only for basic skill test scores, ensuring that schools will de-emphasize developing the noncognitive skills that employers demand.

Better Indicators

A conflict between what we say we want from schools and the near-exclusive focus on academic tests has been growing for decades. A National Academy of Education report in 1987 put it this way: "At root here is a fundamental dilemma. Those personal qualities that we hold dear — resilience and courage in the face of stress, a sense of craft in our work, a commitment to justice and caring in our social relationships, a dedication to advancing the public good in our communal life — are exceedingly difficult to assess. And so … we are apt to measure what we can, and eventually come to value what is measured over what is … unmeasured. The shift … occurs gradually. … In neither academic nor popular discourse about schools does one find nowadays much reference to the important human qualities. … The language of academic … tests has become the primary rhetoric of schooling."

In response, a panel convened by the U.S. Department of Education urged that instead of relying only on the National Assessment of Educational Progress to measure how well schools were doing, the nation should develop a new indicator system that balanced all skills students should have. The panel urged the government to measure learner outcomes such as tolerance, comprehension of pluralism, self-direction, responsibility and commitment to craft, among others. But the panel’s recommendation has gone entirely unheeded.

Econometric studies confirm what employers tell us, showing that noncognitive skills are a stronger predictor of future earnings than are test scores. Adults apparently earn more if they attended higher-spending elementary and secondary schools, even if their test scores were not higher. That more school spending raises students’ later earnings, independent of their test scores, may result from spending on non-academic activities that develop economically valuable character traits. It may result from higher adult-student ratios that help children mature more responsibly without having a direct effect on test scores. Yet because we spend so little effort trying to understand the development of noncognitive skills, we simply can’t be certain what activities in school help to enhance them.

Students with more years of completed school earn more than those with similar scores and less attainment. Test scores explain only about a fifth of the relationship between more schooling and higher earnings, leaving about four-fifths to non-tested qualities that students apparently gain in school.

Among semi-skilled production workers, for example, those who stayed in school longer, again comparing only those with similar test scores, are less likely to quit jobs and so are more valuable to employers. Perhaps young people with perseverance are more likely to stay in school and also more likely to stay on the job so schooling itself may not entirely cause their reliability. But in this, perseverance is no different from literacy. Although reading skill stems from a combination of innate ability, home environment and schooling, we nonetheless think it appropriate to hold schools accountable for literacy achievement. If perseverance is also due to these combined factors, schools should expect to develop student perseverance to a similar extent, while acknowledging that perseverance is also partly innate and partly learned from parents.

Other evidence that attainment matters more than achievement comes from high school dropouts who obtain equivalency diplomas, or GEDs, by taking tests in academic subjects typically studied in high school. If employers mostly valued cognitive skills, adult earnings should be similar for high school graduates and for GED-holders who had the same scores on a common test.

But, in fact, comparing GED-holders who dropped out with high school graduates who have 12 years (but no more) of education, graduates earn more than GED-holders whose test scores are the same. Completing the last year of high school and getting a diploma is worth an increase in earnings that is 10 times as great as increasing test scores by a full year’s worth of learning. Employers value diplomas for reasons that go beyond academic skills that graduates acquire.

Army Attrition

This is not simply a "credential effect" — employers mistakenly thinking that diplomas signify that graduates know more than dropouts. The induction practices of the Army provide good evidence that earning a diploma reflects actual traits that are superior to those of dropouts with similar test scores.

Because of its poor experience with GED-holders, the U.S. Army accepted them prior to 1991 only if their minimum test scores were higher than those the Army required of high school graduates. Still, although GED-holders may have had better cognitive skills, GED-holders’ attrition from the Army was nearly twice that of high school graduates. So beginning in 1991, the Army changed its policy to take only recruits with regular diplomas. It would not accept GED-holders no matter how good their scores.

In 2000, the Army reversed course again, but with even higher qualifications for GED-holders than before. Now the Army accepts GED-holders, but only if they score much higher than graduates on a cognitive test, as well as even higher still on a test the Army administers to assess motivation and reliability. Like employers, the Army seems to value skills not captured by academic tests.

Self-confidence is apparently one noncognitive trait that predicts labor market success. A survey conducted 35 years ago asked young men questions to determine whether they liked to be challenged and whether they felt they could control their destinies. Twenty years later, those with higher scores on these challenge and control measures earned more. For young men of similar social class characteristics, scores on the challenge and control scales were better predictors of future wages than how many years of school they completed or their literacy test scores.

A similar study used a 1980 high school questionnaire to determine whether students felt they could do things as well as most other people or felt proud of themselves. A decade later, those who had high self-esteem in high school were earning more than those with similar academic scores but less self-esteem.

Self-Esteem’s Role

Findings like these don’t tell whether schools did or could have done something to increase young people’s acceptance of challenges, beliefs that they can control their own futures or pride in their accomplishments. But if legislators and school boards expected schools to develop higher scores on assessments of these traits, schools would do more to enhance them.

Some education critics mock schools for trying to teach self-esteem rather than concentrating exclusively on academics. Certainly some programs to teach self-esteem have been silly and were excuses to avoid challenging students, especially lower-class students, to do their academic best. Children should not be encouraged to take pride in unworthy accomplishment, but neither should they be permitted to feel that their worthy accomplishment is not of value. Doing well on standardized tests is, as critics of self-esteem programs insist, one way to increase self-esteem. But it is not the only way.

Schools should be expected to enhance student beliefs that they have enough control over their environments that their actions have real consequence. In academic programs, counseling and non-academic activities, schools should be expected to help children learn to take appropriate pride in their own contributions. If, in an obsessive drive to raise test scores, schools drop concerns about self-esteem to stress only academic learning, students’ later success may be impeded.

Other limited information suggests how schools might strengthen character traits that contribute to adult economic success. One finding has been that workers with similar cognitive test scores, years of schooling and socioeconomic characteristics earn more if their homes are neater. Of course, many highly productive and high-earning workers are sloppier than those who are less productive and earn less. But on average, workers with neater homes earn more when other characteristics are similar.

Certainly schools will not make a big impact on labor market outcomes by urging students to keep neater homes. But neatness and related organizational skills can be taught — the military certainly knows how to teach such habits. If neatness of homes is a proxy for a broader set of attributes and if this finding suggests that organizational skill contributes to adult success, there is much schools can do to design activities that improve such skill.

Worker Traits

Another curious phenomenon, long familiar to economists, is that taller men earn more. Every extra inch of male adult height is associated with a 2 percent wage gain. This could have either of two explanations. Employers may discriminate in favor of tall workers or tall people may have other characteristics that make them better employees. Apparently, the latter explanation is the truth — employers don’t discriminate in favor of tall people because they are tall, but only because tall people happen to have other traits, aside from height, that employers value.

How do workers develop such traits? One clever analysis took account of the fact that young people grow at different rates, spurting at different ages, so although most tall adults were tall as adolescents, not all were. Researchers compared the earnings of tall workers whose growth had come at different points in their development — for example, workers with similar adult heights but different heights at age 16. The researchers found little wage advantage for tall adults who were not tall as teen-agers. This makes it less likely that employers discriminate in favor of tall men because employers are typically unaware of their workers’ teen-age heights.

The earnings advantage apparently stems from the fact that tall teen-agers acquired other characteristics that shorter teen-agers did not. If schools could do better at developing these traits in all young people, graduates would probably be more productive in the labor market.

Self-confidence is again a likely explanation. Taller boys are more popular (among both male and female peers). They also are more likely to participate in athletics than shorter boys, and sports contribute to confidence, teamwork ability and discipline. Adults who were high school athletes earn more than non-athletes who had similar test scores. There’s not much that schools can do to make short students more popular with their peers, but they can encourage all children to take part in team activities and ensure that participation is acknowledged and rewarded. Programs during and after the school day, like debate teams, drama clubs, school newspapers, band and orchestra can build similar confidence and discipline for students who don’t participate in sports. Such programs are now being de-emphasized by schools, which feel they need the time for more academic drill and test preparation.

Filmmaker Woody Allen knew something about what we want from young people when he said 80 percent of success is showing up. Thomas Edison said genius is 99 percent perspiration. Americans read an Aesop’s fable to their children about tortoises and hares, and a more modern story about a "little engine that could," both of which stress the value of character and perseverance. If we truly believe these morals, we should want to know not only about schools’ test scores but about the character traits of their graduates.

Employers use a variety of tests to assess noncognitive abilities or character traits. Among them are tests for integrity that ask job applicants about the propriety of taking small items home from work or personality tests that ask applicants about how often they enjoy taking chances.

To those of us who are not psychologists, it seems applicants might give what they expect is the right answer rather than a truthful one, and so the tests would be invalid. Yet it turns out that such tests predict future job performance fairly well. And scores on such integrity and personality tests are not at all correlated with scores on tests of literacy and math. Other employer noncognitive tests, such as those for teamwork ability or work habits, also don’t correlate highly with tests of cognitive skill. And while there is a big difference between mean scores for black and white students on cognitive tests, there seems to be no black-white gap on job applicants’ integrity test scores.

Employers must decide what balance of cognitive and noncognitive skills they seek. If employers have test instruments that can assess both kinds of characteristics, there is no reason why schools can't develop them as well. This does not necessarily mean they should be used to evaluate individuals. But schools should expect satisfactory group performance on noncognitive as well as cognitive traits and should use such assessments to guide the design of curriculum.

Many educators have been influenced by the work of Theodore Sizer who urged the evaluation of student academic skills by performances rather than by standardized tests. In these performances, students give oral as well as written presentations of artistic and academic projects and answer questions from adult experts. Some school systems have tried to use such evaluations, and during the last two decades controversy has arisen in Kentucky, New York and Vermont, among other places, over whether schools should be permitted to substitute such academic performances for standardized academic exams as graduation requirements.

Essential Skills

But debate about the value of these performances has been poorly framed. It is not simply a question of whether performances do as good a job as standardized academic exams in assessing academic knowledge. Rather, it is whether such performances are necessary to assess communication skills, confidence, discipline and the ability to adapt to unforeseen situations. These skills are as important, if not more so, than academic knowledge.

The Toronto, Ontario, school board announced last year it would issue employability certificates to eligible high school students who did not pass high-stakes academic exit exams. To earn certificates, students will need passing evaluations in practical problem solving, response to criticism, teamwork, time management and other non-academic skills. This is a big step forward and, if the scheme succeeds, the Toronto schools will be the first to attempt such noncognitive assessments. But it is too bad that the Toronto board sees this as an alternative to a regular diploma. All students should meet an appropriate balance of academic and noncognitive skill requirements for graduation.

Richard Rothstein is a visiting professor at Teachers College, Columbia University, and a research associate of the Economic Policy Institute. E-mail: rr2159@columbia.edu. This article is adapted from his recent book, Class and Schools, which includes bibliographic references relating to this article.