Data discovery is changing as result of new data usage patterns and organizational requirements, and is increasingly breaking out on its own as a separate and distinct discipline, according to John O’Brien, principal advisor and CEO, Radiant Advisors, who will present a talk at Data Summit 2016 titled “Enabling Governed Data Discovery in Modern Data Architectures.”
“One of the things we have been seeing in the last year is a separation of data discovery from data visualization and visual discovery. The data discovery path has become almost a category on its own within the big data analytics or big data/BI world,” said O’Brien. In fact, the trend is so strong that his company has added a new advisor in the data discovery and data preparation space to work full-time on the category, which the company is calling “modern data integration.”
There is a need today to focus on how discovery as a capability goes beyond business intelligence and big data or data science, he explained. “Companies really need to hone their discovery skills because there is so much new data available, including social data, external data, and new data sources, both internal and remote. It is not just about giving the business what it needs, but about what can be done with the data.”
To get the most from data discovery, people in an organization need to have access to all of the data, to communicate, collaborate, “and then actually raise the value back into the enterprise,” said O’Brien. “First, it is a capability, and second, it is about the process of changing the enterprise culture to enable users so that we get a massive amount of value by a massive amount of the people, not just a few data scientists.”
However, he noted, as a counterbalance to giving users the capabilities and the platform to enable access to all the data and supporting collaboration with peers, there is also the need for the data management side of the organization that manages risk, and for governance as far as rights and privileges to the information. “But we have to govern in a new way,” he cautioned.
“We are saying that discovery needs to have new governance policies about granting access to people quickly, and it has to have peer reviews because that will be faster than data owners,” said O’Brien. “One of our favorite questions is: what do you do after a discovery has been made? Well, governance needs to step in and say yes, we validated this and the proper usage is within your department, within your division, or throughout the enterprise. You don’t want discoveries to spread and have everyone using their own version of the truth.”
O’Brien said that he hopes that Data Summit attendees will come away from his presentation with an understanding of three main ideas. “One, is I want them to understand that discovery is a capability unto itself; it stands by itself, it is not part of everything else - although it is related. Number two, I want to communicate the importance of having a culture that adopts this, the capabilities of this new generation of data-centric companies, and why it is important at a cultural level - and to more than just a few users. And, finally, we want to be very prescriptive in giving attendees a clear path forward to adopt this and actually bring value into the enterprise.”
O’Brien present his talk as part of a session titled “Enabling Data Architecture,” on Tuesday, May 10, at 10:45 am.
Data Summit will take place May 10-11 at the New York Hilton Midtown with preconference workshops on May 9.
To register, go to www.dbta.com/DataSummit/2016/Register.aspx.