↬ I'm Ivan Krstić (@radian). This is a personal site; I speak for no one but myself.

First OLPC deployment: now it’s real.

This week, Uruguay became the first-ever real, non-pilot deployment site of OLPC XO laptops. And I was there to hand out the first one.

A bit of backstory
Early on, when talking to countries interested in OLPC laptops, we heard one serious concern repeatedly: theft. In places where the price of an XO is several times larger than the annual per capita income, the laptop is an obvious target. To mitigate the issue, the Bitfrost security platform features a three-prong theft deterrence system; without going into too much detail (more information is available), one key protection feature deters XO theft in the delivery chain by deactivating the laptops when they leave the factory. Until they’re activated at the target school with keys that are delivered out of band, the laptops are non-functional bricks.

A couple of months ago, we were picking apart the deployment details of this scheme in a security review with Nicholas and Walter. The scheme requires certain logistical information to be available: either we have to know where laptops are heading somewhat in advance so activation keys can be generated and delivered, or trusted deployment teams have to scan laptop barcodes once they arrive at each school. I was given good assurances that enough information will be available to allow everything to proceed smoothly, but in attempting to establish the worst-case scenario, I might have asked “how can we be sure?” one time too many. Eventually, Nicholas responded with “well, we’ll just mail you along with the first laptop shipment, and then you can be sure just how things will work.” And thus, when Uruguay officially became the first country to place an order for XO laptops, OLPC purchased a ticket that had me landing in Montevideo, the capital, the same day as the first batch of machines. My mission: make sure nothing breaks. Translation: when something breaks, fix it.

The fun begins
The OLPC deployment here is being run as part of Proyecto Ceibal (Ceibo is the national flower of Uruguay), a presidential initiative to equip each child with a laptop. The Ceibal offices are housed in a Montevideo complex called LATU, or Laboratorio Tecnológico del Uruguay, which is a public/private sector cooperative technical lab now responsible for much of Uruguay’s technical certification and quality control programs, as well as serving an incubator role for various engineering and technical projects.
Fiorella Haim unboxes first deployment XO

Due to an almost comical mixup at Carrasco International Airport, my hosts from Ceibal and I were unable to immediately pick up the laptops when they arrived. While waiting for a resolution, I spent time familiarizing myself with various aspects of the deployment, starting with connectivity and servers. (Left: Fiorella Haim, Ceibal technical lead, unboxes the first Uruguay deployment XO.)

Connectivity and servers
Uruguay is using three kinds of access points for blanketing the deployment schools and villages. The Wavion WS410 provides heavy-hitting coverage with six radios and six beam-forming antennae, while custom-assembled Mikrotik R52 routerboards are used as school access points. The Mikrotik devices allow up to three radios and antennae to be plugged in easily, accommodating quite a bit of variation in school size. Finally, BelAir 100 nodes are used for meshing. Schools too small for even a one-radio Mikrotik AP are using Linksys WRT54g devices. All schools have Internet access through DSL, and Ceibal struck an interesting cooperation with the water and cellular utility companies which lets them plant wireless access points on existing water and cell towers. In the village of Cardal, for instance, there’s a Wavion AP atop a cell tower, with a point-to-point link to the school roof which terminates at the school server connected to DSL.

IBM won Ceibal’s server bid, providing x3105-series tower servers. These are dual-core 1.8GHz Opteron 1210 machines with 2GB RAM and two 160GB SATA drives. Much to my joy, they are running Debian 4.0 “etch”, and providing DHCP, Dansguardian-filtered web access, and various monitoring services.

The laptops came from the factory preloaded with OLPC’s Ship.1 software build, and Uruguay requested to be given pre-release access to our upcoming Ship.2 build. We granted the request, and after some testing, Ceibal decided to preload the Ship.2 beta on all machines being handed out.
Guille Spiller flashes some XOsSince network-based upgrades weren’t a possibility in this concrete case, the machines had to be flashed with the new software by hand. Ceibal had also written some custom code they wanted to include on the laptops, and I spent some time cleaning this up and rewriting parts to ease inclusion in the system. After a day, I had put together a cryptographically-signed Ship.2 beta build, a laptop acting as a wireless activation server, and an installer for Ceibal’s custom bits. Everything in hand, we headed to the airport where the laptops were to be reflashed in a rented container in the cargo area. (Right: Guille Spiller, Ceibal technical staff member, checks the progress of an XO being flashed. Photograph inside a container, MVD airport, cargo area.)

We flashed about 200 machines. Were you wondering how long it takes to open a box of 5 XOs, pull them out of their plastic bags, put them on a table, insert batteries, connect them to electricity, open them, insert an upgrade USB key in each, turn them on, let them upgrade the system software then the firmware, wait for them to wirelessly activate then boot, switch to a terminal, mount the USB key, run the installer, power off the machines, remove the batteries, then re-box the XOs?
Reboxed XOs ready for shipping
Why, I’m glad you asked! On average, 12 minutes, with a negligible standard deviation. (The nice thing about the process is that it parallelizes easily with very few people, as no user interaction is required during the actual upgrade steps.) With machines prepared, we were going to head out to departamento Florida at 8AM the next day to do first deployment.

Florida and Cardal
The first deployment machines were handed out in Escuela No. 109 in Florida. The second batch was handed out in Escuela No. 24 in Villa Cardal, which has been a pilot site since May of this year; here we were giving kids mass-produced XOs and collecting their old β2 units. I wrote a bit of software to collect usage statistics on the β2 machines but haven’t yet had a chance to play with it; a quick look at the data seems to indicate that in 6 months, kids created on average 1200 files or about 30-50MB on each machine, much of it writing and photographs from the built-in camera.

For my own part, it’s been a bit stunning to see the deployments and hand out the first machine. I’m not an optimist, and while the OLPC team put all their blood, sweat and code into this project over the last couple of years because of the unshaken belief that it’s the right thing to do, it was only while standing in Escuela No. 109 that I had my “holy shit” moment. A week in summary, if you will: “now it’s real.”

I’ll let photographs tell the rest.

Florida arrival with volunteers
Arrival at Escuela No. 109 with the volunteers who are helping distribute machines.

Kids trickle in
Children trickle in.

The only good use of windows
The only good use for windows: getting into a crowded room to pick up an XO.

First box arrives
First box arrives.

About a thousand machines were given out in three days. Another seven thousand will be given out in Uruguay in the next two weeks, and another 90 thousand in the coming months. Peru is next: they signed an order for 260 thousand today.