Video produced by Steve Nathans-Kelly
A modern analytics lifecycle is a seven-step process, pretty straightforward, and the key is that it takes a perspective from the business user. If the goal is to enable somebody in the business, what you want to do is be able to remove the friction so the process can be done frequently.
In his Data Summit Connect 2021 keynote, Radiant Advisors' John O'Brien discussed how the pandemic and associated challenges of 2020 impacted the analytics lifecycle and drove a shift toward agile business intelligence.
The seven steps described by O'Brien are: identifying the business need, discover the data, connect to the data, do data prep, develop models, visualize and collaborate, the deploy and operationalize.
"You want to see if you can remove the technology friction and have more of a platform where somebody can sit down and go through all seven steps, or go from step one to two, one to four, one to six, and then start over iteratively as fast as they can. So that's our guide."
But what Radiant found after 2020 was the fact that if it wasn't just a business analyst, but maybe it was a business analyst and a data engineer, they saw in sprint meetings that companies would work together to define the business needs. With context it was possible to discover what's the appropriate data.
But when you come into engineering data pipelines, your data engineer could do that. Visualize, collaborate back to the analyst, looking at the output, and then, continuous integration and deployment, monitoring data governance—that was your data engineering team, you know, doing this work. "So an agile sprint, which we saw quite often running two weeks, a portion of that was done by the analyst team and a portion of that was done by data engineers working together., and it made the process go faster. Same with data science development. We call it lean data science development. But once again, a business working with data scientists to build training datasets, integrated datasets, that pull things together to meet the needs. But you still need a data scientists to analyze the models and say, this is the appropriate one. And yeah, there's autoML and other things out there, and maybe you develop multiple models. Maybe you want to put those out in production and monitor them. So MLOps or ModelOps is really taking off in this year."