If you were one of the millions of taxpayers queuing up at the post office to pay off Uncle Sam yesterday, you've got a pretty good idea about the amount of data government must capture, store and analyze each year. And if you're one of those who may have shaved too much of the truth off your return, you probably have some notion of the risk buried in all that data.
The scale of this data—what we now call big data—is growing exponentially, and it affects business as well as government.
Because government has a little more clout when it comes to wringing out this kind of information and is subject to more stringent rules about transparency, that's a pretty good place to begin.
To find out how government can balance big data’s risk and opportunity, Meritalk, the government's online community for IT professionals, surveyed 17 government and industry big data leaders. The new report, "Big Data, Big Brains," for the first time aggregates the thoughts of these big data leaders – providing insights on tools government will need to respond to big data as well as the risks and challenges agencies may face.
The big data leaders say it is the point at which the traditional data management tools and practices no longer meet the demands of the size, diversity and pace of new data. The leaders also say big data is more of a management and process issue than it is a technology problem. Based on panel insights, big data is "the set of technical capabilities and management processes for converting vast, fast, and varied data into useful knowledge," according to surveyor.
The opportunities are significant. Big data leaders say their organizational and IT counterparts can expect four vectors of insight from big data. First, big data will deliver a full, start-to-finish model of a problem. Second, big data is real-time – providing immediate feedback on management decisions. Third, it offers a level of detail drawn from the whole population rather than inferred from a sample. Finally, it offers an understanding of how elements relate to one another, including a clearer picture of causalities.
"Agencies are at a crossroads where big data can be a threat or a great opportunity," said Mark Weber, president of U.S. Public Sector for NetApp. "To ensure big data is an opportunity, agencies need to get prepared by partnering with the private sector on technology solutions that will enable them to easily and efficiently manage, process, and store their data."
According to another joint government report on big data, while the promise is strong, most agencies are still years away from using it. Just 60 percent of IT professionals say their agency is analyzing the data it collects and less than half (40 percent) are using data to make strategic decisions.
Reach BusinessNewsDaily senior writer Ned Smith at firstname.lastname@example.org. Follow him on Twitter @nedbsmith.Follow us @BNDarticles, Facebook or Google +.