Businesses are drilling down through petabyte after petabyte of data in order to better understand their customers. That's a lot of data — there are a million gigabytes in a petabyte, the content equivalent of 13.3 years of HDTV video — which is why this data drilling push is known as "Big Data." Yet less than half of corporate big data initiatives collect and analyze valuable external data such as social media, a new study shows.
When companies don't analyze this data, they're overlooking invaluable insights into their consumers — every 60 seconds, 600 new blog posts are published and 34,000 tweets are shared, according to a global survey on more than 1,100 business and IT professionals conducted by IBM and the Saïd Business School of the University of Oxford.
Part of the problem is the pedigree of the data, the survey found. Business are often suspicious of the sentiment and truthfulness expressed on social networks and question their ability to trust comments, reviews, tweets and other forms of online opinions. Companies place greater trust in internal data, which is the most mature and well-understood data available to them.
Another significant part of the problem in analyzing social media data is the nature of the beast itself —the gnarly world of messy, unstructured data often does not neatly fit in traditional databases such as words, sensor data, geospatial data, audio, images and video.
For most companies, social media and other data sources remain unanalyzed because of a skills gap. Only 25 percent of survey respondents said that they have the required capabilities to analyze highly unstructured data, a major inhibitor in extracting the most value from Big Data.
This is a big deal for companies because the business opportunities and benefits of Big Data are clear. Nearly two-thirds (63 percent) of the survey respondents report that using information, including Big Data, and analytics is creating a competitive advantage for their organizations, a increase of 26 percentage points from the 37 percent who cited it as a competitive advantage in a 2010 study.
By adding social media and other external data to the mix, companies will be able to exponentially increase the amount of consumer insight they are able to extract from Big Data, the study authors say.
"Most companies recognize the potential for Big Data to improve decision-making and business outcomes across the enterprise," said Michael Schroeck, a global information management leader with IBM Global Business Services. "What they struggle with, however, is how to get started on their Big Data journey. Across industries and geographies, the survey found that organizations are taking a pragmatic approach to Big Data. While the majority of them are still in the early stages of adoption, leading organizations are beginning to derive significant value from their Big Data initiatives."