From self-service data preparation and real-time data streaming, to the Internet of Things and Artificial Intelligence, the world of big data and analytics continues to evolve.
Data lake adoption is on the rise, Spark is moving towards mainstream, and machine learning is starting to catch on at organizations seeking digital transformation across industries.
All the while, the use of cloud services continues to grow across use cases and deployment models.
Innovative new technologies are providing the opportunity for budding entrepreneurs and established enterprises alike to develop new business models and processes that change the old way of running operations and improve how customers are served.
Greater speed, flexibility and scalability are common wish-list items for data management infrastructures today. At the same time, the need for smarter data governance and security strategies is also intensifying.
DBTA recently held a roundtable webinar with Clive Bearman, director of product marketing, Attunity; Greg Davoll, VP of product marketing, Alteryx; and Clarke Patterson, head of product marketing, StreamSets, who discussed upcoming challenges and opportunities in 2019.
Data lakes are at a tipping point, Bearman said, since AI and machine learning is everywhere. In the meantime, enterprise data lake objectives haven’t changed: data lakes provide analytics-ready data in near real-time, gives agility to modify processes and technologies, and it maintains or reduces the cost of traditional data warehousing.
But gaps are driving new requirements including the need for automated data pipelines, integrated platform for data lakes, cloud, and streaming, effective and ubiquitous metadata, and multi-data zone methodology.
Attunity for Data Lakes is a solution to watch in 2019, Bearman said. The platform provides streaming data pipeline automation from ingestion to analytics-ready data sets.
Because of data scientist scarcity, the modern end to end analytics platform is on the verge of change, Davoll said.
The modern end to end analytics platform should include discovery, collaboration, prep, analyzing/modeling, sharing, scaling/governing, and deploy/manage.
Davoll believes predictive analytics is the next step in the evolution of the end to end analytics platform.
According to Patterson, data drift is the silent killer of data analytics. Data drift is the unexpected, unannounced and unending changes to data structure, infrastructure, and semantics.
Data Analytics is being transformed by AI, self-service, edge-to-cloud, and the need for speed. This digital transformation requires DataOps, Davoll said.
An archived on-demand replay of this webinar is available here.