AI, machine learning, and edge computing may be all around us, and these technology endeavors all have one important thing in common: Their success depends on the quality of the data fed into them. Data managers recognize that data quality efforts must be improved to meet these new demands and they are concerned about the quality of the data moving through their enterprises. Eight in 10 organizations’ data quality efforts are lagging or problematic.
These are among the findings of a new survey of 238 data managers conducted by Unisphere Research, a division of Information Today, Inc., in partnership with Melissa. A majority of these data executives, 71%, are in positions to either recommend or implement data quality solutions. Respondents represent a range of industries and company sizes.
The 27-page survey report, “Building A Culture of Trust in a Competitive Economy: 2021 Survey on Data Quality,” can be accessed here.
Data quality issues can stem from problems with manual data entry errors, OCR (optical character recognition) glitches, ambiguousness in data, incomplete fields, redundancy, and inconsistent formats. The survey found that managers consider data quality essential to corporate performance going forward in the digital economy. Data quality ranks at, or near, the top of enterprise priorities, and 87% see it as “important” to “somewhat” of a priority, with half citing it as their top priority.
How important is data quality to your organization’s data strategy?
- Top priority to ensuring the ongoing success of projects/initiatives: 50%
- Somewhat important among competing priorities: 37%
- Data quality is often an afterthought at our organization: 8%
- Data quality is never considered: 1%
- Don’t know/unsure: 4%
At the same time, close to half of data managers, 47%, said they either don’t have an active data quality strategy or aren’t aware of one. This suggests that strong data strategies are not as widespread as they should be in an era in which every company needs to compete on data analytics. Data quality approaches may include documented requirements and rules for measuring success, and ongoing activities tied to business goals. Such strategies are critical, as customer and product data are on the line.
While there generally is confidence in the data that moves through their organizations, this confidence is lukewarm. Most data managers are at least somewhat skeptical about the quality of data in their enterprises. At the same time, they recognize achieving data quality as a top corporate priority. Less than one-third of respondents, 30%, indicated they are “completely confident” in the integrity, accuracy, and trustworthiness of the data that moves through their organizations.
How confident are you in the integrity, accuracy, and trustworthiness of data at your organization?
- Completely confident: 30%
- Somewhat confident: 64%
- Not confident at all: 3%
- Don’t know/unsure: 4%
Data quality initiatives are not widespread enough to enable consistent quality, and the discovery of issues is often through informal processes. The greatest risk factors to data quality come from project complexities and other unknown factors that may arise as innovative technologies and architectures are implemented. Gaining organizational support is another challenge.
For a majority of respondents, 74%, the quality of data presents issues, at least some of the time. For 26%, the problem is severe, cited as a “constant, ongoing issue.” Only 21% could say data quality is a non-issue at their enterprises at this time.
Human error may be the leading cause of data quality problems. A majority of respondents, 58%, indicated employees are manually keying in enterprise data, while 49% reported their customers are entering data.
There are many ways that data quality glitches are surfaced, including formal and informal processes. For 45% of respondents, such issues are uncovered sporadically as they upgrade databases or move to new systems. Likewise, 43% are alerted to issues as new analytics initiatives are launched. For another 45%, such issues are uncovered on a more regular basis through data quality systems and tools. However, disturbingly, 43% report finding out about such issues through customer and partner complaints.
Many enterprises do not attend to data quality issues on a consistent basis. For a large segment, 40%, such projects are few and far between—having only taken place more than a year ago, if ever. Similarly, another 40% report having had a data quality project within the past year. Having a process for acknowledging and addressing data quality issues improves the ability to respond to customers, design and release products, and operate at peak performance.