Reliance on digital data is unseen danger

The accumulation of data is not inherently a bad thing, but what happens if you lose it and it can't be recovered?

For most people, the crash last week at the Nasdaq stock exchange in New York means very little. A glitch in the software that publishes the prices of stocks caused a failure that led Nasdaq to shut down for two hours, meaning thousands of US stocks could not be traded. One estimate suggested $5.7trillion in shares were locked up during that time.

In the real world, such stories have little effect on the minds of the public. The technical language of finance is a world away from the experience most people have with money. But as the 2008 global financial crisis showed, what happens with obscure financial instruments can have implications for ordinary shoppers in the malls.

Moreover, the same type of error that caused Nasdaq to shut down also led to other large companies losing money or having their services disrupted. The algorithms that run in the background of many major companies can go wrong and when they do, the knock-on effects can be catastrophic, especially as these algorithms can go wrong in fractions of a second.

In just the last 10 days, Google suffered a four-minute meltdown, Amazon went down for almost an hour and Microsoft's email servers had a glitch that took three days to resolve. These outages can seem small, but for those people who could not access their emails or personal documents, they are more than mere inconveniences.

What lies behind all these outages are two linked trends of the modern world. The first is the accumulation of Big Data - vast amounts of information about what we do and buy, where and how we travel, and a limitless number of other examples - and, second, the automation of responses based on that data. A majority of trades on Nasdaq are automated and tens of thousands are carried out every minute, without human oversight. On Amazon's website, products are recommended based on the aggregated opinions of tens of thousands of people.

That does not mean that such data or automation is inherently bad. The examination of such data can mean traffic flow improves and medical research is swifter. But it does point to a lesser-known reliance on computer networks and digital infrastructure that are as frail as other infrastructure. After all, if your electricity stops, you call the electricity company and they fix it. But who do you call if your email is inaccessible, or the software on your smartphone shuts down, or all your online photos are deleted? And what happens if that data can't be recovered?