While data quality tools find issues in production, it is too little, too late. Production data issues are expensive to fix and cause business discontinuity and reputation risk. iceDQ believes companies should adopt a left-shift approach to prevent faulty processes and data from entering production.
Sandesh Gawande, CTO, iCEDQ, discussed “Data Testing: A Left-Shift Approach to Data Quality” during his Data Summit 2023 presentation.
The annual Data Summit conference returned to Boston, May 10-11, 2023, with pre-conference workshops on May 9.
iCEDQ is an ETL Testing platform, designed to identify any data issues in and across structured and semi-structured data. Because of its ability to identify the data issues, it is used to automate ETL testing, data warehouse testing, data migration testing,?business intelligence report testing, big data testing, and production data monitoring.
The problem is there are different costs associated with different stages within the experiment lifecycle, Gawande explained.
The idea is quality assurance and quality control equals total quality. However, the quality assurance part is a very proactive approach that focuses on the processes that generate the data.
Quality control identifies defects in the product before delivery. It’s reactive and focuses on the product, which is data.
“If you take care of the quality assurance part, you get rid of everything in the quality control space,” Gawande said. “I’m not saying you don’t need quality control, you need to focus on the assurance for 99% of the process.”
People don’t talk about quality assurance because it isn’t “fashionable” enough, he noted. Think DataOps by reducing data defects in production, reducing project timelines, improve trust in data, and complying with regulations.
Many Data Summit 2023 presentations are available for review at https://www.dbta.com/DataSummit/2023/Presentations.aspx.