DEMOCRATIZING AND ELEVATING
Today’s tools “are designed to be used by anyone capable of asking meaningful questions,” Leong observed. “For example: Who are the most influential people in my organization? Who knows what? How does my company feel? These are just a few examples of today’s business-critical questions that can be addressed with minimal to no data science expertise. A fundamental difference in this approach is that insights come not from manipulating database tables but rather from scanning information created by humans for humans.”
With the plethora of tools available to address various aspects of the data lifecycle, companies are finding it more expedient to move to more all-encompassing platform approaches, be they SaaS or PaaS. This is accelerating a shift away from IT-driven data management and enablement to user-driven approaches, said Peter Jackson, chief data officer of Exasol. “These solutions are aimed at data management and business users rather than IT teams, and users are claiming ownership of the platforms and applications as they are no longer the exclusive domain of IT.” What it means is that business users without deep technical expertise can more easily use these tools, and this has been pivotal in helping to accelerate their productivity, he added.
This next generation of tools “is bringing instant visibility to large datasets that we never had before,” said Aubrie Cunningham, senior vice president of business intelligence and pricing at MedRisk. “This type of transparency allows business leaders to make informed decisions based on real time, and even predictive, data. Businesses are greatly benefiting by catching problems before they happen. They are becoming much more efficient in presenting data, eliminating the need to translate what the data is saying. Dynamic visualizations and alerting features allow for a new level of oversight for leadership.”
Data enablement platforms “have evolved significantly over the last decade,” pointed out Kunal Shah, senior product manager at SAS. “As the volume, velocity, and variety of data increased, so did the need for new versions of data platforms that were capable of storing large amounts of both structured and unstructured data in a centralized repository. These platforms—referred to as data lakes—focus on facilitating prescriptive and predictive analytics. And as cloud adoption increased, a new generation of data platforms was created primarily around cloud data storage and management, specifically around a cloud data warehouse.”
New approaches to data management and enablement are also changing the nature of data managers’ jobs. A common feature seen across many of today’s data solutions is greater automation, freeing data managers from repetitive, and often overwhelming, rote tasks. “Data professionals spend most of their time on manual processes to ingest, clean, and transform data in support of data operations,” said Chris Bergh, CEO of DataKitchen. “Automating these processes slashes maintenance costs and enables data scientists and engineers to focus on analytic insights that address business challenges.”
Add to the mix the increasing volume of low-code and no-code tools in the market. Until recently, “the people employed to run and support data management tools had to be highly skilled developers and data scientists,” said Paz. “With the recent rise of DevOps, data management is shifting to the hands of people with standard software skills. We’re now seeing a trend of low-code and no-code data management tools, utilizing a simple drag-and-drop canvas. So, businesses are becoming less reliant on highly skilled data scientists, and these experts can focus on developing innovative new data management concepts and processes.”
These types of tools not only appeal to citizen developers but data users as well, said Tricot, noting that the tools are being created to address specific audiences. “These tools are more specialized and enable companies to grow their hiring pool and focus on domain experts instead of focusing on domain experts who have deep technical skills. They enable teams to be up and running faster.” The tools not only appeal to citizen developers and analysts but also professional developers and data engineers as well, he added.
Bergh sees an emerging leadership role for DataOps engineers, who are essentially DevOps engineers that oversee the data pipeline moving from ingestion to analytics. “If we think of data operations as a factory, then the DataOps engineer is the one who owns the factory assembly line that builds a data and analytic product,” said Bergh. “DataOps engineers introduce automation into a data organization that can improve the productivity of data scientists and analysts by seven to 10 times.”