Choosing the Right Technology

How to sort through claims of computer-based software designed to boost reading performance by Charol Shakeshaft, Dale Mann, Jonathan Becker and Kara Sweeney

 Your school board has charged you with increasing reading scores by one extra month in grade-level gains. Your options are these: Reduce class sizes from 21 to 15 students per class (requiring additional teachers) or add instructional technology to the reading program.

Both methods will yield the same outcome—a month's increment of learning—according to Lewis C. Solmon, senior vice president and senior scholar at the Milken Family Foundation, who analyzed data from our study of learning gains in West Virginia. But reducing class size and hiring more teachers is significantly more expensive than implementing a technology program. “The total cost figures,” Solman says, “translate into $636 per student per year for class size reduction and $86 for West Virginia’s Basic Skills/Computer Ed (computers).”

Assuming you opt to add instr-uctional technology to the reading program, the question is how do you sort through the array of technology-based programs for reading instruction and determine which is best suited and most economical for your schools and district? Not everything that’s available is worth the cost, both in money and time.

We have spent the past 10 years evaluating the impact of instructional technology on the reading and math achievement of children in hundreds of schools nationwide. What we have learned about how technology improves reading achievement can inform the questions school administrators should ask before making purchasing decisions.

What Needs Doing?

If you are considering purchasing and implementing an instructional technology program aimed at improving reading achievement, do some research first. We are surprised how often school districts purchase instructional technology programs without reviewing data about their effectiveness or considering whether they really need the program in the first place.

Take inventory. What are you doing now and how well is it working for different students? How many children are reading below grade level? What’s happening in the classroom? Are teachers actually using the approved reading curriculum?

If the current repertoire is meeting the needs of all students, if students have positive attitudes toward reading and if the current approach is efficient, don’t spend more district money on reading technology.

However, if your students’ reading achievement could be improved, technology may help. It’s time to take a closer look at what is going on in the classrooms and determine whether the district is ready to act.

Is the problem with younger students or older ones? Is phonemic awareness an issue? Comprehension? Vocabulary? All of these?

Is the current system the most economical instructional approach, both in terms of cost per student and time spent by student and teacher? Is your district ready to make a change? Do teachers agree that additional resources are necessary? Do parents understand the goals? Have all stakeholders been involved in identifying needs and discussing a solution?

Do you have the resources to implement what you buy? Resources include not only the funds available to buy the software or subscribe to an Application Service Provider, but also a sufficient number of computers in the classroom, comprehensive technical support, professional development opportunities and the good will to sustain new materials and change in the classroom.

Take-Home Technology

Learning can happen anytime and anyplace, and technology is mobile. Is this a lab or classroom activity? Will students be learning at home?

Some instructional technology products supplement what teachers are already doing in the classroom. While skill-and-drill computer-related programs are often derided, they are generally more fun and more responsive to a student’s needs than are the ubiquitous paper-and-pencil worksheets. When learning is fun, students are more engaged in the process.

Programs that supplement the curriculum with CD-ROMs that the students can take home help boost reading skills by extending the learning day. For example, Lightspan Inc.’s Achieve Now!, is a K–6 standards-based learning program on CD-ROM that students use in school and take home to “play” on PCs or Sony PlayStations with their parents. In our four years of research at three school sites, we have documented reading gains of up to 8 months for children who extend their learning at home. Thus, e-learning technology can involve parents as partners in the learning process.

Some programs are quite comprehensive. Scholastic’s READ 180, for example, which targets low-performing students in the middle grades, is a combination of instructional reading activities using individualized adjusted instruction and support in word study, vocabulary, comprehension and spelling; modeled reading with the classroom teacher; independent reading of selected books; and teacher-directed instruction.

Do Your Homework

Hundreds of instructional technology products promise to help students read better. It is up to school decision makers to sort out vendors’ claims. Several organizations screen instructional technology products and provide information about research and results to make it easier for administrators and teachers to estimate success in their settings. (See resource list.)

As you read the reviews and the research, note where and how the research was conducted. Does the study population resemble yours? How was the technology implemented in the study and how difficult will it be to replicate that implementation?

Are there comparisons between students who did and did not use the technology? If not, did the researchers use analysis of covariance, gain scores or value-added techniques? Do the studies measure outcomes that are educationally or politically important, such as state test scores, nationally published standardized achievement test gains?

When considering a product, determine how the materials map to your local and state learning literacy standards. Most companies state that their materials are based on the National Council of Teachers of English reading standards, but you must determine how easily teachers can use those maps in lesson planning.

The vendors themselves should provide information about who is using their products. Contact these customers and talk with them about how they are using the products and what results they have seen. Remember, teacher enthusiasm does not always translate into increased achievement. Ask for hard data.

Available software ranges widely, from materials that merely reproduce pages from textbooks to truly interactive, individualized learning experiences. We have examined the claims of many advertisers in the Kappan, Education Week and in the exhibition halls of education conferences. More often than not, no reliable data backs up their claims of improved reading scores for students who use the product.

If the software will supplement or reinforce learning, teachers will need to know how to choose materials for particular students. Do the products provide diagnostic information for individual students? Progress reports? Multiple approaches to instruction such as graphics, sound and video? Does the student receive tutorial and meta-cognitive feedback? Can the student track learning gains? Can parents? Can administrators?

One way to determine the cost of a technology-based curriculum is to compute the cost of the materials per student plus training time and any hardware that must be purchased and maintained. Using these totals, compare how much the same gains would cost using traditional methods.

Using our example of Lightspan materials, we can compare two methods of achieving reading gains on the 4th-grade New York State English Language Arts test. If we look at studies of effect size and gains of other methods, we find that reducing the class size to 15 produces the same gains as using the company’s Achieve Now! Comparing the cost per student of all of the expenses in implementing and maintaining the commercial product (software, hardware, technology support, training) with the cost of new teachers (teacher salaries and benefits) the comparison in cost over three years in one of our research sites is $289 per student using a technology solution and $1,362 per student by reducing class size. This comparison doesn’t take into account any additional classroom costs that would be necessary to reduce class size.

Adequate Infrastructure

School districts often separate instructional technology from curriculum on the organizational chart. You end up buying learning solutions consistent with instructional and pedagogical goals but inconsistent with the IT infrastructure.

For example, a Web-based program might match the instructional philosophy of the district. However, if schools and classrooms have significant connectivity problems, the likely result is frustration and insufficient implementation, if any.

Also, reading technology program vendors should supply the district with clearly defined documentation of the minimum necessary technological specifications. How much memory and what processor do the workstations need? Do workstations need to be networked? If there is a Web component, how broad does the connectivity need to be (T1, T3, etc.)? Will the program work with any Web browser (Internet Explorer, Netscape, etc.)? Do plug-ins (QuickTime, Real Player, etc.) need to be installed on the workstations?

Technical support is also essential. What technical support does the vendor provide? Will the vendor train the district technology support personnel or the teachers? Will the technical support team come to you?

Another option is an Application Service Provider, which houses the programs on an off-site server. Arizona has contracted with LearningStation.com to provide students with Internet access to 250 educational programs from the classroom or from home. Because the programs are housed on a server in Tempe, Ariz., all students need to connect to this library of opportunity is a 56K modem. Upgrades are instantaneous, technology representatives don't have to touch every desktop in the school, security is increased, and the threat of viruses is reduced. Think about renting, not buying. Subscribing, not purchasing.

Even with a range of programs and sufficient hardware, teachers won’t use reading technology if they don’t know how or are not comfortable with it.

Involving teachers in the selection of the software from the beginning and then providing continuous professional development maximizes their buy-in and their effective use of the technology. Help lines, technical support by phone, online and in-person instruction are all essential.

Professional development is necessary, but not sufficient to ensure smooth implementation. Teachers won’t always report a problem with the software or the computer. Therefore, the administrator or technology resource person should check with teachers weekly to determine whether they are having problems with the software or hardware or questions about their use.

Implementation Keys

Learning gains are directly related to successful implementation of the technology (if it is not used, it will not have an effect). When we evaluate the effectiveness of a product or advise a district on implementation, we track the presence of 12 factors. The more factors that are present, the deeper and wider the implementation and the greater the achievement outcomes.

* Sound adoption process: Adoption requires that the organization be ready to field new strategies and that the resources necessary to adopt or adapt these strategies are available.

* Student body stability: The more similar the student body in attitudes, backgrounds and beliefs, the easier it is to implement new technology. This doesn’t mean districts with mobile or heterogeneous populations shouldn’t introduce technology solutions, only that these factors require additional planning.

* Administrator support and involvement: While it is helpful to have support from the central administration, schools and even individual teachers can initiate change and can resist and re-direct change. Leadership counts.

* In-school support and involvement: Schools where technology implementations are most successful are those where teachers lean on each other for support and advice—especially with respect to the technical aspects of the program.

* Community pressure, support, apathy: Community pressure and support for literacy change help stabilize technology implementations.

* Continued professional development: After implementation, provide additional support and training to maintain teachers’ commitment and help them develop the skills necessary to refine, integrate and renew the program.

You can't lead an orchestra if you don't know the music. Superintendents and other administrators need training as well.

* Integration into school structure, curriculum, policies, rituals, culture and supervision: Schools are more likely to adapt the product to the school than vice-versa. Therefore, products whose use requires extensive changes in school or classroom structure, paradigms or processes are less likely to produce intended outcomes than those that require only modest change.

* Ongoing evaluation: What gets counted gets done. Programs that are evaluated are implemented. Teachers frequently raise the quality of implementation simply because they know someone is paying attention. Further, data gleaned from evaluations can help highlight successes and inform changes along the way.

* Policy priority: In most states, reading and language arts are more than rhetorical priorities. Their instruction is supervised, monitored and enforced in many instances with career consequences for school employees. High priority, official programs are usually more faithfully implemented than are others.

* Teacher commitment, ownership: Most research indicates that teacher commitment is a necessary condition for the successful implementation of school innovations, including technological ones. Teacher buy-in maximizes the prospects for change.

* Teacher rapport with students: The relationship teachers have with students may not predict the fidelity of implementation, but it may affect student outcomes. If students are more willing to credit the instruction of teachers they respect and who understand them, they may more fully trust the teacher’s message. In turn, this may enhance the effectiveness of a program.

* Teacher subject matter expertise: In general, the more the teacher knows about the subject matter, the more likely the teacher is to be able to use the product effectively. We also have found that good technology sometimes substitutes for poor teaching or lack of expertise.

A Careful Choice

Research is key to choosing the right instructional technology reading program for your district. You must decide how you want to use the technology for classroom learning, determine whether the product you are considering matches the needs of your students and teachers, whether your school’s infrastructure is compatible with the software and hardware and what technical support the vendor and the district will be able to offer.

Support and professional development should also be made available to teachers. When teachers feel comfortable operating a technology program, they are more likely to use it regularly and more effectively than those who are not.

Reading and literacy technology programs offer many more and additional advantages than traditional methods for students. They are less expensive and more effective than some traditional teaching methods. However, determining which technology-based reading program best suits your district’s needs is critical for positive results.

Charol Shakeshaft is managing director of Interactive Inc. and a professor of foundations, leadership and policy studies at Hofstra University, 11 Stewart Ave., Huntington, NY 11743. E-mail: charolshakeshaft@aol.com. Dale Mann is managing director of Interactive and a professor emeritus of organization and leadership at Teachers College, Columbia University. Jonathan Becker is research director of Interactive and a doctoral student of politics and education at Teachers College. Kara Sweeney is project director of Interactive and a graduate student in the English program at SUNY Stony Brook.