We live and work in a "datafied" world, say the authors of a new book that takes the measure of the new world of Big Data. There's more of it, it's messy and it asks us to accept correlations rather than causation. It can tell us "what," but not necessarily "why." And it's going to change how we understand our world.
Viktor Mayer-Schönberger and Kenneth Cukier have been exploring the implications of what we now commonly call "Big Data" since it was just a volcano spewing petabytes of structured and unstructured data from data warehouses, call-center records and desk drawers. The genie was out of the bottle, but we didn't know what it was saying.
Mayer-Schönberger is professor of Internet governance and regulation at Oxford University and the author of "Delete: The Virtue of Forgetting in the Digital Age." Cukier is the data editor of The Economist and a prominent commentator on developments in Big Data.
Today, they say, we now have access to the tools and understanding to begin to make sense of this brave new world of Big Data. Mayer-Schönberger and Cukier have collaborated to create a comprehensive guide to this new data landscape of data in "Big Data" A Revolution That Will Transform How We Live, Work, and Think" (Houghton Mifflin Harcourt, 2013).They recently shared insights from the new book with BusinessNewsDaily.
BusinessNewsDaily: What is your working definition of Big Data?
Viktor Mayer-Schönberger /Kenneth Cukier: We resist giving a concrete definition since that would limit it. But basically, it refers to the idea that we have so much more information these days that we can apply new techniques to it, to spot useful insights or unlock new forms of economic value. There are things we can do with a large body of data that we simply couldn’t when it was in smaller amounts.
BND: What do you mean when you say that the world is being "datafied"?
VM-S/KC: Often, Big Data has been portrayed as the consequence of the digital age. But that misses the point. What really matters is that we are taking things that we never really thought of as "informational" and rendering them into data form. We’re "datafying" it, to coin a term. For instance, we have datafied location and text. Facebook has datafied our friendships and things we like. We are even datafying human posture and an individual’s gait. Once it is data, we can use it, process it, store it and analyze it and extract new value from it.
BND: You write that Big Data allow the data to speak for itself. Please explain.
VM-S/KC: Before Big Data, we used the comparatively little data we bothered to collect to answer a particular question we had. But as a result for new questions we often had to collect data again. When we have close to all data available, we can ask a wide variety of questions without having to recollect the data — and we have much more detail available. Yet the idea of listening to the data also means being willing to consider what the data says, even if it challenges our classical understanding of things.
BND: In understanding Big Data and its uses, how important is it that we understand of the concept of "good enough"?
VM-S/KC: Because Big Data analysis is often based on data that is messy and of varying quality, it will give us a sense of direction rather than results that are exact down to the inch, the penny, the atom. But what we lose in accuracy at the micro level, we gain in insight at the macro level. And this new sense of direction lets us answer practical questions in a timely way — in that sense, it is very often simply “good enough.”
BND: How does Big Data represent a major change in the way we understand the world?
VM-S/KC: Many of the decisions we make are less "empirical" than we think, less driven by data, because the cost of collecting, storing and processing it used to be so high. But as the economics of harnessing data changes, we are able to apply it to new areas. For example, researchers are analyzing the real-time vital signs of premature babies to spot the onset of infection 24 hours in advance. In health care, we haven’t tapped data in this way before — though now that it’s feasible, it’s obvious that we should. The researchers are essentially comparing the data "signatures" that predict an infection — which is a correlation; it says nothing about the underlying biological mechanism at play. But harnessing Big Data forces us to change our understanding of the world by accepting correlations — the "what, not the "why.". Credit: Cover of Big Data image courtesy of Houghton Mifflin Harcourt
BND: You write that Big Data represents three shifts in the way we analyze information. What are they?
VM-S/KC: We term these shifts "more," "messy," and "correlations." It means that in the Big Data age, we have more data available relative to the question at hand than ever before. But some of that data is inexact and of varying quality, so we need to embrace a modicum of messiness. Together, more and messy data enable us to see connections in the data we would never have guessed. These correlations tell us the "what," rather than the "why," but often that is sufficient for us to act. For instance, if we know that babies exhibiting a particular combination of vital signs are likely to develop an infection 24 hours later, we can stop the likely infection earlier — and that saves lives, even though we do not know the exact underlying biological cause.
BND: Will even small organizations and groups be able to exploit Big Data?
VM-S/KC: The infrastructure for Big Data analysis — the capacity to store and process large amounts of information swiftly — is now widely available through cloud services. Small organizations can lease such capacity at low cost exactly when and for how long they need it. No initial investment is required. This makes big data analysis available for a wide variety of groups and startups without the need for steep initial investments.
BND: How will Big Data benefit us?
VM-S/KC: Big Data will benefit every sector of society. It will make business more efficient, and permit them to develop innovative new services based on the reuse of data. It will power self-driving cars, empower consumers in predicting the future price of products and services. It will change — by individualizing — health care. And it will bring about a revolution in education, as we shed light on which educational methods — for instance what textbooks — work and which don’t.
BND: What are the dangers of Big Data?
VM-S/KC: Protecting individuals’ privacy is a challenge in the age of Big Data as many data users will want to use personal information for multiple purposes. But a new danger emerges: propensity – the use of Big Data analysis to hold individuals responsible for acts they are only predicted to commit. In the extreme this may negate human volition — our ability to decide freely whether and when to act. Punishing people for predicted rather than actual behavior is undoing the very notion of fairness in our society.
BND: The algorithms of Big Data and Predictive Analytics are getting increasingly better at figuring out what we like. Will this kill creativity and serendipity?
VM-S/KC: Only if we follow data blindly — and get trapped into fetishizing data, imbuing it with more meaning than it deserves. To avoid this we must ensure that in the era of Big Data we carve out a space for the human, for irrationality, creativity and intuition, for at times going against the data.
Reach BusinessNewsDaily senior writer Ned Smith at firstname.lastname@example.org. Follow him on Twitter @nedbsmith.Follow us @BNDarticles, Facebook or Google +.