Moving to the cloud may seem, on its surface, a good way to better manage the quality and viability of data resources for enterprises. However, as data accelerates its move to the cloud, the complexity involved in assuring quality and viability has only increased.
This is one of the findings of a new survey of 224 IT, data managers, and professionals conducted by Unisphere Research in partnership with Melissa (“Data Quality Challenges and Strategies for the 2020s,” Unisphere Research, December 2022). The survey finds awareness of the importance of data quality is on the rise, but that managers struggle even more with gaining budget and organizational support than before.
Gaining funding for data quality projects is increasing as a challenge, though data quality is vital to data-driven enterprises. The survey revealed that cloud complicates, not eases, data quality matters.
For many data managers, cloud provides opportunities to offload the rote or onerous tasks associated with data management, provisioning, and even security.
However, data quality and integrity must, by necessity, remain within the purview of enterprise customers. If anything, moving to cloud has increased the issues associated with data quality and integrity.
The growth of data in the cloud has been significant, the survey shows. Just about every enterprise stores data in the cloud. For 46% of enterprises, this constitutes a majority of data (50% or greater) now maintained in the cloud—up from 32% just 1 year ago.
The complexities introduced by cloud to managing data quality have reportedly increased. Forty-six percent report increased challenges to their data quality efforts, up from 42% in the previous survey.
There’s an urgency to data quality and integrity. Business needs to be data driven, and the data that’s driving things needs to be of the highest quality attainable.
Data quality efforts are on the rise, the survey shows, as 58% report to have a data quality strategy—up 5 points from the previous survey. In addition, there has been a significant rise in organizations recognizing the importance of data quality to their data strategies—56% consider this to be a top priority, up from 50% in last year’s survey.
The survey also finds some measure of greater confidence in data integrity—33% now express “complete” confidence, up from 30% in the previous survey. While there’s greater confidence in the quality of enterprise data, managers admit it’s more of an issue than it was in the previous survey. More than one-third, 35%, see it as a constant issue that needs to be dealt with, up 9 percentage points from a year ago.
Financial challenges to data quality are increasing and may be limiting the frequency of remedial efforts, the survey shows. At least 37% are struggling with calculating ROI for data quality investments. At least 32% are challenged with getting the right amount of funding, up from 28% a year ago. This is seen in specific budgetary constraints as well.
Data quality initiatives remain too far and few between, the survey shows. While there is an increasing emphasis on data quality, the survey finds increasing budget constraints, and this may be limiting the frequency of data quality projects.
Respondents in this survey are almost evenly divided in terms of the frequency by which they engage in data quality efforts. For a large segment, 40%, such projects are scarce—having only taken place more than a year ago, if ever. Similarly, another 40% report having had a data quality project within the past year. This is unchanged from reported frequency in the survey a year ago.
This survey shows that data quality initiatives are not widespread enough to enable consistent quality, and discovery is often through informal processes.
The greatest risk factors to data quality come from project complexities and other unknown factors that may arise as new technologies and architectures are implemented.