These days, data managers have an embarrassment of riches when it comes to the platforming, storage, and provisioning of data and associated analytics applications. However, selecting the right environment for pressing business needs can be confusing and overwhelming. The key is understanding how to map the right technologies to the business problems or opportunities at hand.
The array of choices include classic, tried-and-true data warehouses, which have evolved with the times, now joined by data lakes, which offer a means to rapidly deposit data of all types, and, lately, data lakehouses, data fabric, and data mesh. Underneath it all are cloud-based platforms and services providing compute power, capacity, and supporting automation.
Data warehouses are still omnipresent—most companies (82%) actively support and invest in a data warehouse; a recent survey of more than 200 IT leaders and professionals finds.
The "Data Delivery and Consumption Patterns Survey", Unisphere Research, a division of Information Today, Inc., May 2022, also finds emerging modern data architectures including data lakehouses, data fabric, and data mesh components are being deployed to support leading-edge initiatives such as artificial intelligence, machine learning, and the Internet of Things. Close to half of enterprises have data lakes already implemented, one in five use data lakehouses, followed closely by the adoption of data mesh and/or data fabric solutions. Just about all are being deployed on the cloud. Current users of emerging data architectures—data lakehouses, data fabric, and data mesh—intend to keep investing heavily in these initiatives.
“The traditional ways of looking at data—the mindset of ‘I want a data warehouse; I want a single-purpose analytics collection’—doesn’t apply anymore,” said Steve Zisk, senior product marketing manager for Redpoint Global. “Because now all data has valuedepending on the use case.”
WHAT BUSINESSES WANT
Business demand and preferences for analytics has shifted toward these modern data architectures—and for good reason. “Data is being generated at a mind-blowing rate and businesses are starting to understand that data now defines business operations,” said Traci Curran, director of product marketing at Actian.
In addition, “expectations for experience are extremely high,” said Scott Gnau, head of data platforms for InterSystems. “Everyone wants to leverage and monetize all this new data, coming from smart devices and everywhere else. Businesses are realizing that the insights they can pull from existing systems are instrumental in making informed and timely business decisions. But traditional data management and analytics architectures make leveraging all of this siloed data very difficult. Thus, businesses are turning to more integrated approaches, such as data fabric architectures.”
However, industry observers also caution against the mindset of simply investing in technology for technology’s sake. Instead, there needs to be an emphasis on “value, ROI, reuse—deeper and wider impact on business decisions—instead of amassing as much data as possible for no clear purpose,” said Sebastian Werner, AI evangelist at Dataiku. He also urges—and sees—greater decentralization and democratization on data analytics, versus “bottlenecked centers-of-excellence."
This is accompanied by demand for simplicity and easy access. “Most users prefer simple to configure or ready-made dashboards, and don’t want to get too deep into the underlying technology that makes their data available to them,” said Casey McGuigan, product manager at Infragistics.
Often, the challenge to business is free and clear access to data. The “consumption” side of data has been outpacing the “supply” side, as it is often hampered by regulations that constrict access to data. “When it comes to data analytics, organizations are caught in the middle, between the business need to analyze more data faster that is streaming in from multiple sources, with the competing need to better control access to and govern that same data,” said Sophie Stalla-Bourdillon, chief privacy officer for Immuta.
Whether being harnessed for operational management, or additional revenue opportunities, “data is now a lighthouse for every business,” said Curran. “Every department wants to access data to make improvements and foresee potential troubles. This has resulted in a desire for self-service data access but has also opened risk as data becomes more distributed.”