Organizational issues such as governance and skills—not technology requirements—are the greatest challenges that IT and corporate managers are facing in the emerging world of big data. To get a better handle on the complex new world big data is catalyzing, executives and professionals recognize they must reimagine and re-architect the concept of the “data center”—and what ultimately is coming out of may be a surprise to everyone. These are some key takeaways from a recent survey of 319 corporate and IT managers, conducted by Unisphere Research, a division of Information Today, Inc., in partnership with Cloudera and Intel. (“Key Shifts: Transforming the Data Center,” March 2016)
Managers see big data—defined as greater data volumes, data types, data sources, and data velocity—as having many potential benefits, including a better understanding of their customers, products, and business operations. While there is boundless enthusiasm—and hype—about big data, few enterprises have initiated ongoing big data projects.
Significant barriers that stand in the way of launching big data initiatives include the fact that projects are complex and require time to implement, and that there is also a need to develop a successful strategy to realize the benefits of big data investments. This demands new types of tools and platforms, and more importantly, the creation of clear business objectives to guide such efforts. Plus, survey respondents indicated that they believe creating an appropriate data pool from disparate data types and sources is difficult, and they are concerned about whether they have the necessary skill sets internally to accomplish such tasks.
Many companies have not yet launched a big data project, the survey shows. Fewer than 15% of enterprises have ongoing big data efforts underway, while a similar percentage have no plans at all to enter the big data arena. It’s not for lack of data resources—the survey shows many organizations have a significant amount of data under management and data volumes are growing at a very healthy pace annually. The lag in initiatives to leverage this data may be due to the fact that data growth presents significant challenges, particularly in terms of security and data governance, as well as data access.
At what stage are you in your big data initiatives?
Preliminary discussions – 34%
Active planning – 21%
Piloting – 12%
Implementing – 8%
Currently in use – 13%
No plans to use big data – 12%
Analytics opportunities represent the most compelling business case to pursue a big data initiative, the survey confirms. Close to one-third of respondents state that analytics will drive their big data initiatives, followed by one in four seeking greater agility in running their businesses.
What is your most compelling reason to consider a big data project?
Improve capacity for advanced analytics – 31%
Increase business agility – 24%
Meet end user demand – 20%
Improve operational efficiency – 18%
Lower IT costs – 5%
Other – 2%
While there are many benefits that organizations can achieve through advanced analytics, survey respondents indicate that there are technical, professional, and cultural hurdles in their paths. The greatest technical hurdles for advanced analytics are data access and the management of data from disparate data sources. The survey also found executives and professionals were struggling with determining the best ways to create and position data pools to facilitate their efforts.
What are the biggest data management issues posed by big data projects?
Difficulty merging disparate data sources – 42%
Difficulty accessing appropriate data from disparate data sources – 40%
Difficulty transforming data into usable formats – 36%
Concern about data quality – 34%
Difficulty in establishing data governance rules – 27%
Difficult in determining data access – 13%
Other – 1%
Of course, technical data management complexities make up only one set of vexing issues. Many organizations do not believe they have the appropriate skill sets internally to be successful with big data. Indeed, the lack of internal skills is by far the biggest impediment, cited by three in five respondents.
What are your biggest barriers to initiating a big data project?
Lack of skills internally – 59%
No clear business case –33%
Rapidly changing ecosystem – 29%
Inadequate hardware infrastructure – 24%
No clear technology use case – 19%
Lack of C-level sponsorship –18%
No end user demand –14%
Other – 4%
The survey finds a majority of organizations are overseeing transformations of their data centers to meet these new realities. Nearly 60% of the respondents have systematic plans for data center transformation that involve either ongoing upgrades or an annual menu of new projects. Such efforts are being propelled by a desire to engage with new applications and the need for the data center to be more nimble. Other drivers include the need to cut costs, the ability to benefit from new technology, and the desire to improve business agility.