Real Time
As IoT gains hold, the delivery of real-time data to assess real-time problems or predict imminent complications will soon become commonplace. Real time has become a vital part of the data integration equation. Fifty-seven percent of managers and professionals in a recent Unisphere-IOUG survey state that there is now strong demand for delivery of real-time information within their organizations. Less than one-third of respondents, however, say they are capable of delivering most of their data in real time at this time (“Moving Data at the Speed of Business: 2016 IOUG Survey on Data Delivery Strategies,” February 2016). The challenge for data managers will be to work with the business to identify what parts of the enterprise or which datasets should be configured for real-time delivery.
Hadoop and Spark
Hadoop and Spark are part of the open-source wave that continues to offer cost-effective and useful capabilities to enterprises of all sizes. Being open source environments, they provide a cost-effective and highly scalable means to package and deploy large and varied datasets. While there has always been some form of “big data” in existence—“big” is a relative term—organizations have long had networks of sensors, devices, embedded applications, remote systems, and log files providing significant pools of streaming data.
Prior to Hadoop, the capture and analysis of datasets of any kind required proprietary tools, and was an expensive and resource-intensive undertaking. Hadoop—and the open source ecosystem that accompanied it—has made big data analytics an extremely cost-effective option within reach of everyone.
The challenge to data managers will be acquiring the skills needed to build out these open source environments.
NoSQL and Cloud-Based Databases
Databases that can be quickly deployed and accessed for online functions are enabling an agility unseen within many large enterprises, while providing growth platforms for small to medium-sized businesses. A survey of 300 data managers and professionals, conducted by Unisphere Research and sponsored by Dell Software, finds NoSQL technology is being used or being deployed at 21% of enterprises.
Another 19% of respondents say they plan to implement NoSQL within the next 1–2 years. One-third of respondents expect NoSQL to have a significant impact on their database operations in the next 3 years. Hadoop is being used or is being deployed among 20% of respondents’ companies (“The Real World of the Database Administrator,” March 2015).
The challenges for data managers are integrating these new forms of databases with existing relational database management system environments and enabling the seamless transfer of data between the two.
Virtualization and Modernization Solutions
The ability to plug legacy systems—and their silos of data—into evolving big data analytics networks will unleash significant amounts of untapped enterprise information. The modernization wave keeps accelerating, with the deployment of virtualization solutions and approaches to enterprise systems that abstract the underlying legacy systems within an accessible service layer.
A recent survey of data managers finds that close to half, 45%, report they have virtualized their data environments. In addition, close to two-thirds, 62%, report that more than half of their mission-critical data is included in their virtualized environments (“2015 IOUG Data Protection and Availability Survey,” August 2015).
The challenge for data managers is addressing the performance issues and additional complexity that arises as virtualization expands across their enterprises.
Data Discovery
Data discovery opens up business intelligence and analytics to business end users through visualization and easy-to-use interfaces. The emerging technology reorders and reorganizes the way data is managed, and provides a great deal more flexibility than traditional BI and analytics tools. Data discovery is inherently designed and built for users, enabling them to sift and sort through datasets.
The challenge for data managers is maintaining the integrity and security of data before it is streamed to business users’ workstations.
Blockchain
Blockchain—which is essentially a distributed general ledger maintained in a highly distributed fashion across the internet—is most commonly associated with the Bitcoin virtual currency. But ultimately, it can function as a global database for a range of applications and interactions, from all manners of finance to maintenance agreements. Blockchain may be in its nascent stages, but it has the potential to disrupt many aspects of data management and information technology, as it is community maintained and verified.
The challenge for data managers at this point is to learn how blockchain works, and probe vendors to see what progress is being made in making this a mainstream solution.
The Changing Enterprise
Cost management is still a driving concern of data executives, managers, and professionals. However, attention has also turned to being able to deliver new capabilities which reach customers in new ways, and delivering new forms of value to oorganizations.