A highly agile data environment built on containers, microservices, DevOps, and DataOps means data environments can “rapidly scale up and down, not just according to operational demands of the application but also according to demand for data analytics,” said Chart. “Current capacity and scale are restrictions of the past when it comes to analytics possibilities. Now the ability to ‘burst’ capacity as needed eliminates many of these barriers and frees the business to take advantage of time-sensitive opportunities previously unavailable.”
The need for high-level analysis, productivity, and implementation “has never been greater,” said Pete Brey, product manager, hybrid cloud object storage for Red Hat. “Agile infrastructures and updated data services to unlock new capabilities have taken hold. Through new methodologies such as DevOps, microservices, and containerization, organizations can build, test, and deploy applications quickly, efficiently, and more frequently.” In addition, he said, these methodologies help enable agility for data at rest, in motion, and in action.
More flexible approaches also support the unfettered delivery of data through a technique called “federated learning or multi-cloud machine learning,” said Dobrin. This technique applies machine learning to situations where data cannot or should not be moved due to concerns about data privacy, secrecy, regulatory compliance, or simply the size of data involved, he explained. “The process allows data professionals to access previously siloed sources of information, save large sums of network egress fees, and ultimately retain complete control of their organization’s data.” The use of federated learning empowers businesses “to pull deeper insights from their increasingly complex data environments without sacrificing data privacy and security,” said Dobrin.
Containers will dominate database choices in the coming months and years as well. ScienceLogic’s Chart predicted “increasing out-of-the-box support for databases running in Kubernetes, to the point where it will become the default deployment model for new applications, where all components—both stateful and stateless—rely on the Kubernetes environment for a secure and available operating environment. To support this, we will continue to see both evolutions of traditional database products play well in this environment and continued development of entirely new databases, purpose-built to rely on Kubernetes for production deployments.”
Inherent Risks
While there are many benefits being realized as organizations move to next-generation agile technologies and methodologies, not all data organizations are ready to embrace the changes. It is important to ensure that when these platforms are adopted, the processes align with the organization’s capacity to best leverage them, said Potter. “Sometimes the technology is ahead of the business’ ability to effectively deploy and reap its benefits. It ends up being considered unsuccessful because there’s an inherent mismatch.”