How ‘Open Data’ will tip us towards task-free era

Most people would agree that we are on the verge of a significant transformation in the global economy, driven by advances in data and analytics.

For some historical perspective, the analytics journey of the last two hundred years has been one of rapidly decreasing costs for information storage and computation enabling new ways of working with data. Until the second half of the twentieth century, while we already had a number of advanced statistical techniques, it was simply too expensive to manipulate information in any but the most straightforward ways.

The word “computer” referred to a person, not a machine, and if you wanted a column of numbers added up, or a graph created, or customers counted, you had to pay someone to do it manually. If you wanted it done again, you had to pay them again.

The rise of electronic computers of course changed that calculation, with prices for simple tasks dropping from (say) ten or twenty cents down to a tiny fraction of a cent. This opened up new fields for analysis – it was now possible to sort a list of numbers (a computation-intensive task), or add them up every week or even every day, or to plot them on a scatterplot (an almost impossible task for large data sets before about 1980).

As the cost of storage and compute continued to fall, we developed new methods of analysing data that took advantage of these capabilities. Starting in the 1990s, statisticians starting using techniques that simply weren’t possible in the old days – working through data sets of millions or even billions of individual points and performing complex transformations on them. This allowed us to solve previously linear optimisation problems and gave rise to the discipline of predictive analytics.

Then, as costs fell further in the 2010s, we started to find ways to work with unstructured data such as images and audio recordings, enabling us to apply algorithms to an ever-growing range of subjects and progressively automate more and more tasks.

We now stand on the edge of an era in which data and analytics will take us even further.

When we combine the ability to process unstructured data, the rise of information-sharing (through initiatives like Open Data – firstly with Open Banking which is planned to go live publicly in Australia next year), more advanced analytic techniques such as machine learning, and ever-cheaper processing and storage, we are on the verge of being able to automate a much wider field of everyday tasks.

Soon, we will reach a tipping point, where the question is not what routine tasks we can solve through technology, but rather which ones we can’t.

For example, while it’s certainly become easier to pay your bills, or book a flight, or pay your taxes than it was thirty years ago, all of these tasks still require you to be actively involved in most of the steps. You still need to put in your search terms and look at different insurance options to book a rental car, and to check your phone bill for any surprises, and hundreds of other similar chores.

I believe, though, that as we move into an Open Data world, combined with much better data and analytics capabilities, humans will be able to step back from many of these routine tasks. An algorithm that understands your preferences will be able to narrow the choices down into a manageable set of options for your review – and with service providers sharing data (with your permission) behind the scenes, they will be able to coordinate to provide you, for example, a handful of holiday options where your flights, rental car, accommodation, and insurance all match up automatically.

So, in the short term, Open Banking will spur innovation and heightened competition as banks and fintechs compete to provide new insights to customers with their data, and as customers shop around for the most competitive offers. But in the long run, it won’t just spur additional competition – it will create an entirely new set of business models and change the nature of industry competition, as banks and others are judged by their ability to work together to provide better business outcomes.

All you’ll have to do is choose where and when you want to go or – again, with your permission – your doctor will be able to see information from your wearables, your grocery purchases, and your gym membership to get relevant information to supplement your medical profile.

There is a huge amount of work to be done to realise this future state. Individual participants in these complex ecosystems will have to invest heavily in technology, data, and customer experience in order to mesh seamlessly with other service providers. And all of this needs to rest on a solid foundation of security, privacy, consent and data ethics to ensure that individuals are protected and have full control of their data.

When these new capabilities come together in an Open Data world, resting on a bedrock of privacy and consent, the benefits to all Australians will be considerable, and the opportunities for organisations such as Westpac – which will be ready for the planned February 2020 go live of Open Banking – to lead the way will be tremendous.

This article is based on a speech delivered at the FST Media Future of Financial Services conference in Sydney today.