There never has been a more interesting time to be involved in the data management field. Data not only has become “the new oil” but is also the catalyst that is powering organizations to new heights of success. The past year has seen the rise of powerful analytics and an embrace of new tools and platforms emerging to more effectively tap into the power that data offers. DBTA reached out to industry experts to document the most important trends shaping data management in 2018.
Rise of the chief data officer
Having a strategic approach to data requires specialized leadership, and this is fueling the growth in the number of CDOs. “As data continues to proliferate, it will only become more important for an organization to have someone directly responsible for the company’s data transformation strategy,” said Kristin McMahon, senior director of product marketing for enterprise information management solutions at SAP. “The chief data officer role will be the linchpin in helping companies bridge the gap between IT and operations, taking an overall responsibility for data management—including overseeing data movement, legitimizing data, and leveraging new, innovative technologies to inspire change and enable the digitization of the customer experience.”
The evolution from batch to open source streaming accelerates
Emerging technologies and approaches are speeding the evolution of many core processes from batch processing to streaming. New “open source data science tools are game changers for both businesses and individuals,” agreed Peter Wang, CTO and co-founder of Anaconda. “Anyone can easily get results from that type of software and turn their data into actionable insights much quicker than before.” Such solutions “include Kafka for capturing and distributing incoming streaming data; Nifi for data routing; Ignite for faster in-memory processing of the incoming data; Hadoop 2.0 for data access and storage; and Kubernetes for managing how we scale a streaming infrastructure which is susceptible to bursts,” said Marc Concannon, CTO of Clavis Insight. All of these technologies are new to many companies and are, “at their core, all about working with more and more data and extracting the relevant insights from this data quicker and hence making it available to customers quicker.”
Software-defined storage
There’s a hardware-side story that goes with today’s generation of data management tools and platforms. “Storage vendors are adopting software-defined storage principles to provide composable storage across multiple systems,” said Douglas O’Flaherty, marketing director with IBM Spectrum Solutions. “This trend will accelerate as organizations add clusters to support AI and cognitive workloads.” In addition, O’Flaherty sees that all-flash arrays are innovating with new technology, greater density, and fast data reduction. “The adoption of scale-out flash storage with seamless tiering, automated archiving, and more features is trending.”
The continued rise of the Internet of Things
The Internet of Things (IoT) may be one of the most disruptive technological advances seen across the data space, experts state. “We’ll see the effects of a hyper-connected universe as IoT becomes ubiquitous,” said Jake Freivald, VP at Information Builders. “Generating vast amounts of information, these sources will potentially provide non-traditional data such as video and audio that must be integrated to provide quality insights. Regardless of the form, the data must be analyzed in tandem with other data sources to offer value. When done correctly, IoT data offers companies unprecedented opportunities to drive efficiencies; optimize labor, energy, and other resources; cut costs; and operate more competitively.”