A new study with very strong credentials from Ofer Malamud (University of Chicago and NBER) and Cristian Pop-Eleches (Columbia University, Harvard University’s BREAD, and NBER):

This paper examines the impact of having access to a home computer on child and adolescent outcomes. To avoid the bias due to non-random access to home computers, we exploit a unique government program which provided vouchers towards the purchase of a personal computer for low-income children enrolled in Romanian public schools. Since the fixed number of vouchers were allocated based on a simple ranking of family income, this program affords a stark regression discontinuity which allows comparisons across students very similar in family income and other respects, but who experienced markedly different access to a computer at home. In 2007, we conducted a household survey of children who participated in the program in 2005. Using these data, we show that children who received a voucher were 50 percent more likely to own a computer.

Next, we show that receipt of a voucher had a large impact on time spent in front of the computer and decreased the amount of time spent watching TV and doing homework. Children in household that won a voucher also report having lower school grades and lower educational aspirations. There is also suggestive evidence that winning a voucher is associated with negative behavior outcomes. Nevertheless, we find that having a stay- at-home mom and the presence of rules regarding computer use do mitigate some of the negative effects of winning a computer voucher, indicating that parental monitoring and supervision may be important mediating factors. …

Both columns 1 and 2 of Panel A indicate a negative effect of winning a voucher on GPA as reported by parents and the children themselves. The effect is statistically significant for the child reports, at 0.36 grade points or about one third of a standard deviation.The decrease in GPA based on parental reports is smaller and not statistically significant at conventional levels. Similarly, parents in households who won a voucher are 13 percentage points less likely to report that their child intends to attend college. … Interestingly, winning a voucher does not increase the intention to major in computer science in college. …

Children in households who received a voucher show a large reduction in their school behavior grade during the 2005-2006 school-year. This result is large and statistically significant … To summarize, the evidence on GPA, college plans and the school behavior grade presented in Table 5 suggests that, if anything, computer ownership has a negative impact on child academic and behavioral outcomes. …

However, despite the efforts of the Romanian Ministry of Education to encourage the use of these computers for educational purposes through the provision of educational lessons, relatively few children have educational software installed on their computer, and fewer still report using the educational software on a regular basis. Instead, our analysis brings out the important role of parents in shaping the impact of home computer use on child and adolescent outcomes. We find that in families where mothers stay at home and where parents have rules regarding computer use, the negative effects of winning a voucher are greatly reduced. Thus, our findings suggest caution regarding the broader impact of home computers on child outcomes. They also raise questions about the usefulness of recent large-scale efforts to increase computer access for disadvantaged children around the world without paying sufficient attention to how parental oversight affects a child’s computer use.

Fascinating. I find it particularly interesting that, despite educational software from the Ministry being available to the parents in the voucher program at no cost, few appear to want it or install it, and few children report using what educational software was provided. Internet access was relatively unavailable for voucher households, however, and I wonder how much that colors the study’s results.

So what went wrong?

I think there was one key error with the choice of hardware, while the software missed the point entirely.

Choosing hardware
No one knows exactly what an education-centric computing device should look like, or even what the “education-centric” moniker actually means. Over time, people have gotten better at reasoning about it out loud, though. What, then, are the defining physical (non-software) qualities of an education-centric computer?

Ownership — it has to be something the kids can own, because communal computer classrooms only tend to work where there are classrooms and teachers to run them, and few enough kids not to compete for computer time. But in the neediest parts of the world, such settings are in the minority, and it’s all the other settings that need help the most.

Portability — the kids should be able to carry the device around as much as they want, at the very least because in many cases, the device will be substituting for books. (It’d be crazy if it didn’t: at OLPC, we found a classroom in an African capital that had three books for 90 children. This isn’t rare, or even particularly uncommon.) More fundamentally, if the device is to encourage “always-on learning” by becoming a brain extension that the kids can turn to whenever they become interested or curious about something, they need to be able to have it with them a substantial amount of time. That means desktops and other clunky, heavy, and immobile solutions are out of the picture.

Proper human interaction — the device should allow rich, prolonged user interaction. It needs to have a sensible input device, for instance, which rules out most cell phones — just imagine typing a history paper on a phone keyboard. And you need a reasonable viewport. Even the iPhone’s display, the reigning champion of mobile phone screens, is hardly one you want to use as your sole window into reading, writing, and browsing the web. So no, “give them cell phones” isn’t going to fly. Anything with a screen below a 6-7 inch diagonal is suspect.

A match for infrastructure and external conditions — the device needs to be compatible with infrastructure realities, which vary wildly with locale. Much of the least developed world struggles with obtaining clean water daily. Those places won’t be helped by computer rollouts at this stage. But one level up, many countries that generally have enough food and clean water still don’t have a ubiquitous electrical grid, and they certainly don’t have wireless access points hanging from every tree. An ideal device for the developing world would suck little power, allow networking without infrastructure in remote locations, and could manage the scorching heat and fine sand grains of Libya, the bone-freezing cold of Mongolia, and the incessant rain and humidity of Peru’s jungles. Extra points if it’s also rugged to survive (un)intentional abuse by children.

Romania’s choice
If these are our criteria, Romania’s effective choice of home-owned, primarily desktop computers for the voucher scheme fails to meet an important one: desktop computers are not portable. This means that computers become segmented both physically and temporally. They can only be used in a particular place, for particular hours in the day. Children can’t take them to school, they can’t benefit from teachers incorporating computers into the curriculum as I’ve seen happen in rural Peru, and without internet connectivity, they can’t work (or play!) with other children when using their machine. Perhaps most importantly, children can’t turn to a non-portable computer every time they get a curious itch — curiosity being the instigator of a tremendous amount of learning at an early age.

Not even the nerdiest of children I know want to willingly carve out “learning time” in their day. But they do get curious constantly, and unless they’re in a position to get answers immediately from an authority figure, most of the time the point of interest will be unanswered and forgotten. Laptops, especially when preloaded with knowledge bases, can change the picture entirely, and I think this was Romania’s key hardware miss. (I should note that the program didn’t mandate desktops, but the economic realities appear to have forced most voucher winners to use their vouchers for purchasing desktop machines.)

The software issue might be more telling. The Ministry provided voucher winners with “530 multimedia educational lessons”, which they hoped would encourage educational use of the machines. In other words, they assumed that children will come home after a full day at school, do their regular homework, and then, by themselves, of their own accord, and with no tangible benefit or reward, do more schoolwork.

Does that sound like many children you know?

Though the study’s point about parental oversight is very well taken, here’s what I continue to think: learning with computers should be always-on. Kids should come to treat them as surrogate brains that are very likely to have answers when kids have questions. They should have interfaces that are collaborative and allow for learning that’s shared, open and involves peers. Where an established schooling system exists, the collaborative aspects of the computer should be used as much as possible in teaching, which is a tall order, and in letting children work on problems together, which isn’t.

This is the promise of Walter’s Sugar Labs, and it’s a good one. The Malamud/Pop-Eleches study is an entirely unsurprising reminder that, as the New York Times found anecdotally, merely throwing computers at children — even if they’re laptops — just won’t do.