Processing big data in real-time for artificial intelligence, machine learning, and the Internet of Things poses significant infrastructure challenges.
Whether it is for autonomous vehicles, connected devices, or scientific research, legacy NoSQL solutions often struggle at hyperscale. They’ve been built on top of existing RDBMs and tend to strain when looking to analyze and act upon data at hyperscale - petabytes and beyond.
DBTA recently held a webinar featuring Theresa Melvin, chief architect of AI-driven big data solutions, HPE, and Noel Yuhanna, principal analyst serving enterprise architecture professionals, Forrester, who discussed trends in what enterprises are doing to manage big data in real-time.
Data is the new currency and it is driving todays business strategy to fuel innovation and growth, Yuhanna said.
According to a Forrester survey, the top data challenges are data governance, data silos, and data growth, he explained.
More than 35% of enterprises have failed to get value from big data projects largely because of skills, budget, complexity and strategy. Most organizations are dealing with growing multi-format data volume that’s in multiple repositories -relational, NoSQL, Hadoop, data lake..
The need has grown for real-time and agile data requirements, he explained. There are too many data silos – multiple repositories, cloud sources.
There is a lack of visibility into data across personas -- developer, data scientist, data engineers, data architects, security etc..Traditional data platforms have failed to support new business requirements –such as data warehouse, relational DBMS, and ETL tools.
It’s all about the customer and it’s critical for organizations to have a platform to succeed, Yuhanna said. Customers prefer personalization. Companies are still early on their AI journey but they believe it will improve efficiency and effectiveness.
AI and machine learning can hyper-personalize customer experience with targeted offers, he explained. It can also prevent line shutdowns by predicting machine failures.
AI is not one technology. It is comprised of one or more building block technologies. According to the Forrester survey, Yuhanna said AI/ML for data will help end-users and customers to support data intelligence to support new next-generation use cases such as customer personalization, fraud detection, advanced IoT analytics and rea-time data sharing and collaboration.
AI/ML as a platform feature will help support automation within the BI platform for data integration, data quality, security, governance, transformation, etc., minimizing human effort required. This helps deliver insights quicker in hours instead of days and months.
Melvin suggested using HPE Persistent Memory. The platform offers real-time analysis, real-time persist, a single source of truth, and a persistent record.
An archived on-demand replay of this webinar is available here.