Data management has never been so unfettered—and yet so complicated at the same time. An emerging generation of tools and platforms is helping enterprises to get more value from their data than ever. These solutions now support and automate a large swath of structural activities, from data ingestion to storage, and also enhance business-focused operations such as advanced analytics, AI, machine learning, and continuous real-time intelligence. However, persistent data and systems integration challenges, along with legacy systems and processes, still stand in the way of efforts to build end-to-end data-driven enterprises.
NEXT-GEN TOOLS EVOLVING
Today’s generation of data management and enablement solutions has come a long way in recent years, and industry leaders agree that the new generation of data enablement tools and platforms is far more advanced than 10, or even 5, years ago. “Today’s technologies are easier to use, more focused on the business user, and designed to reduce the complexity of data silos,” said Dan DeMers, co-founder and CEO of Cinchy. “Some leading-edge tools go even further by enabling data to be originated in a way that prevents new silos altogether. These tools are taking data enablement away from uber-techies and data nerds and truly democratizing the field.”
This shift is spurred by the growth of a “broader audience that is consuming data, from deeply technical users to less technical domain experts,” said Michel Tricot, co-founder and CEO of Airbyte. Now, he added, even “mom-and-pop shops are dealing with large datasets, heterogeneous data, and complex analytics.”
Companies of all sizes and business categories are adopting digital channels and approaches to compete in today’s economy. “Ten years ago, the business world wasn’t truly digitized,” observed Adi Paz, CEO of
GigaSpaces. “Most consumers still went to bank branches, placed calls to agents, drove to the supermarket for groceries, and paid cash for their taxi rides.” As digital has become the dominant channel for many businesses, consumers have also gotten used to real-time, frictionless digital touchpoints.
Accordingly, tools and platforms seek to abstract these digital touchpoints from what are often complex and confounding underlying systems. “These tools are based on innovative architecture concepts designed to decouple apps from their systems of records, bypassing the spaghetti structure that companies already have in place and took as a given reality for years,” said Paz.
Consider how far things have come along in just a few years. Five years ago, “ETL for modern data architectures wasn’t solidified enough to guarantee interoperability and enable full data optimization,” said James Beecham, co-founder and CTO with ALTR. “Organizations have been moving toward a vision of one-click, dynamic data integration and enablement that works for the great cloud migration,?and that path accelerated during the last 2 years.”
Traditional databases only make up a fraction of a company’s digital knowledge. “Much of it lies out of sight, in the mountains of messages, documents, and files a company shares every single day,” said Kon Leong, CEO and co-founder of ZL Technologies. “The new generation of data enablement is solving the challenge of harvesting this data to answer critical questions that traditional structured data analytics is not capable of answering.”
An important emphasis for today’s solutions is that “data is managed based on rules and policies, thus allowing for real-time validation and curation,” said Radhakrishnan Rajagopalan, global head of technology services at Mindtree, who noted that this improves the ability to refine data cataloging and data discovery. “In contrast, old tools and technologies processed data in silos without knowing the context and the needs of the intended audience.”
In addition, the associated visualization and BI tools landscape is evolving quickly, from “tools that mainly accessed highly structured and formatted pre-modeled data to tools that can leverage the variety of data types or formats as well as the massive volumes,” said Balaji Ganesan, CEO of Privacera. “BI tools evolved and soon included built-in performance optimizers and aggregators to pre-calculate analytical insights. Auto data indexing and search style access also became popular because they made tools easier to use and required less expert skills to perform dashboards and analysis.”
The ability to decouple apps from their original systems of record “liberates the immense data previously stored in siloed systems of records,” said Paz. “This enables businesses to deliver a steady flow of new digital services and use cases at a pace they couldn’t even dream of just a decade ago.”