David Jonker, senior director, Big Data Marketing, SAP, recently highlighted the results of a new big data survey, the “2013 Big Data Opportunities Survey." The survey revealed a variety of practical approaches that organizations are adopting to manage and capitalize on big data.
The study was conducted by Unisphere Research, a division of Information Today, Inc., and sponsored by SAP. In the survey report, Unisphere Research analyst Joe McKendrick points out that it is important to separate the hype from the reality for enterprises in their day-to-day business. Contrary to the perception that big data only provides value when it is crunched on the type of large-scale clustering technologies used by web companies, many big data issues are being successfully addressed now by conventional technologies such as relational databases.
While many respondents believe their current technology is capable of helping them manage and capitalize on big data, current solutions may be in need of refresh. More than one-third of respondents, 36%, say faster querying of the data is a challenge they are attempting to deal with. Another 43% are concerned about providing faster access to the large datasets that are proliferating. To address the need for speedier access and analytics, one third of respondents indicate that they are working to extend and optimize their current infrastructure, while one-quarter are adopting new frameworks such as Hadoop and in-memory databases.
Reflecting on the big data study, Jonker said that when people think of big data now it is still largely in terms of how they can deal with more traditional data – transactional data. For many customers, the ability to deal with sensor data or social media data is far out in the future. They are still dealing with large amounts of relational data and not yet uncovering the benefits of unstructured data. “In fact, the research suggests quite strongly that what a lot of people are doing is trying to improve existing processes and the number of organizations that are seeking to untap big data to start whole new business models or embark on new areas of business is relatively small,” said Jonker. “That was a hunch that the data confirmed.”
The second key finding, said Jonker, is that the struggle organizations have with the relational database and how they store data largely revolves around getting to the information faster - collecting the information and getting the information faster. “That came out loud and clear,” he said. With the adoption of technologies such as columnar databases, Hadoop, and massively parallel processing solutions, they are all trying to get around the speed bump and in many ways the data confirmed that, he said. “Mostly, what they are deploying is relational databases and mostly what they have got are speed bumps.”
While it is still batch processing, Hadoop helps with the speed bump in a couple of senses, says Jonker. “If I don’t know what the data is, I don’t want to spend a lot of time, cleansing the data, prepping it to load it into a data warehouse. Hadoop lets you plunk it there and figure it out later.” It also helps in the sense that you can scale out lots of machines that are all processing smaller amounts of data, he adds.
“At SAP, we believe in-memory is the future. Everything that these solutions are doing – data warehouses columnar databases, Hadoop – they are all trying to get around the disk bump; they are all just taking different approaches to getting around the limitations of disk,” says Jonker. SAP’s perspective, he notes, is, why not just mine data in-memory. While companies may have different specialty data stores in their data environments, in-memory can tie them together.
The “2013 Big Data Opportunities Survey” report is available from DBTA at www.dbta.com/DBTA-Downloads/ResearchReports/3905-2013-Big-Data-Opportunities-Survey.htm.