Taking a Measure of the Measure


Imagine for a moment that you have a 12-inch ruler in your hands. As an instrument of measurement, we depend on this ruler. We depend on it to be accurate, valid and reliable. The same would be true for a yardstick, a 12-foot tape measure or other similar measuring devices.

The 12-inch ruler, however, has its flaws. If I need to measure something longer than 12 inches, it is possible for errors to creep in. In fact, if I measure something where I have to lay the ruler end to end several times, the width of the line I use to continue my measurement will soon be a factor in the overall length.
nd if I need to lay the ruler end to end say 100 times, fatigue of the measurer will soon enter the process. So our 12-inch ruler isn’t universally accurate after all. For things that are 12-inches long, it is excellent. Nothing is better. However, measurements of more than 12 inches, and also less than 12 inches, are problematic and prone to estimation and error.

There is an enduring principle of measurement: Measurement freezes the measure and what is measured in place. Don’t believe it? Think about the metric system vs. our system of inches, feet and yards. We are probably the only advanced country in the 21st century that does not use the metric system. And what if the 12-inch ruler was the only measurement tool for length we had? How long would it take before virtually everything became 12 inches long or some derivative of it? My guess: Not very long.

Accuracy Compromised
Let’s go back to our 12-inch ruler. What if what we had to measure was not a flat, straight-line measurement but was circular? Again, we have a valid, accurate and reliable measure in our 12-inch ruler, but it compromises its validity, accuracy and reliability when applied to situations for which it is not intended.

Remember ∏r2? To find the circumference of a circle (the distance around it), you multiply the radius squared times ∏. If you remember from your high school math, ∏ (pronounced “Pi”) is an estimation that has an infinite set of numbers to the right of the decimal place. Once again, we could probably easily measure the radius or diameter but using the algorithm to calculate circumference introduces error even though the tools used are accurate, valid and reliable.

One more example. What if we are trying to find out the distance between Lincoln, the Nebraska state capital, and Chicago. We could use our 12-inch ruler or a yardstick. Or we could use the odometer on our automobile. Which is more accurate, more valid and more reliable?

The odometer is probably least accurate due to being affected by inflation of tires, pavement heat and calibration at the factory. If I set out to find the distance to Chicago using my ruler, how much error will I introduce when I have to lay the ruler end to end, draw a line and repeat it a zillion times? How much error will I introduce as I get tired of bending over and drawing the line to move the ruler one more time?

Given the fact that my odometer is not precisely accurate, the factors of fatigue of the measure and fatigue of the measurer will likely not enter in. Which is the more accurate and reliable measure of this distance? I would opt for the odometer.

The real question here is not how many feet or yards it is between Lincoln and Chicago, anyway. What use is it to know how many feet it is? It may not even be relevant to know how many miles because what I probably want to know is how long it will take me to get there. While I can convert the feet and yards into miles and figure the likely speed at which I will be traveling, the odometer reading is much more useable and much more relevant to the question I am trying to answer: How long will it take to get there?

Dynamic Tools
So what is the point here? So much of traditional measurement is caught up in this analogy. We assume that standardized tests are like rulers in terms of their accuracy, validity and reliability. In some ways they are. They measure things that are “inch-like” or “foot-like” best but do not measure well the things outside of those dimensions.

Maybe we should focus less on the instruments of measure and focus more on the information they provide and how we use the information. Can we trust that the information we get is the ultimate question of accuracy, validity and reliability?

Measuring learning cannot be done with rulers. Measuring learning takes dynamic measures that change with the outcomes to be learned, with the instruction provided and with the students being taught. Measuring learning requires gathering information from multiple indicators of what students know and are able to do before we jump to the conclusion that learning has occurred (or has not occurred).

If we want rote content learning, then we can use our tools of inches, feet and yards to determine whether students measure up. If we want learning that is more about thinking skills, problem-solving skills and information/ technology mastery skills, then our traditional tools won’t work. We need tools to match the things we are trying to measure.

The tools we need the most to measure real learning are most often not pre-manufactured. One does not go to the test store and buy one off the shelf. Dynamic learning requires tools that are built for the purpose and are available on the spot. Who best to build these tools? Teachers! And where’s the best place to build them? Classrooms!

Doug Christensen is commissioner of the Nebraska Department of Education, P.O. Box 94987, Lincoln, NE 68509. E-mail: His column appeared previously in the Nebraska Council of School Administrators’ newsletter.