Cultivating an AI-ready enterprise begins at the level of data. Without a trusted, robust data foundation—and a blueprint for developing one—AI initiatives will lose their technological footing.
Rik Tamm-Daniels, GVP of ecosystems and technology, Informatica, and Manish Dalwadi, director of product management, Databricks, joined DBTA’s webinar, Build a GenAI-Ready Enterprise with Informatica & Databricks, offering their guidance on strategies and tools for preparing data infrastructures for enterprise-grade AI.
With generative AI (GenAI) adoption skyrocketing due to its “sheer transformative potential of generative AI as a technology,” it is becoming widely accepted that data is a key enabler in its success, explained Tamm-Daniels. According to 2024 Informatica research, to overcome AI roadblocks, 78% of CDOs predict their data investments to increase in 2024 with 100% of CDOs planning to specifically invest in data management capabilities.
Tamm-Daniels identified the following as key attributes of a successful GenAI app:
- Grounded to prevent hallucinations
- Contextualized by enterprise information and semantics
- Fueled by high-quality, accurate data
- Easy to develop, deploy, and update without lots of hand-coding
- Governed and secure, transparent and traceable
Ultimately, the success of AI is dependent on robust, responsible, and relevant data. Bad data leads to bad AI, noted Tamm-Daniels, regardless of industry: “It can be spectacularly transformative, but it can also cause a lot of challenges. We do find that having a proper data foundation really does create applications that are more valuable, are safe, and can be deployed at scale for the enterprise.”
To address these challenges, Informatica’s Intelligent Data Management (IDMC) is designed for every aspect of data management to power both analytics and AI use cases. Optimized to manage data regardless of where it lives, IDMC is a cloud-native, as-a-service, microservices-based platform with a security-by-design architecture and consumption-based pricing.
IDMC is the only complete, end-to-end data management solution for Databricks, Tamm-Daniels said, centralizing a variety of data functions—from data cataloging to data integration, data quality, master data management (MDM), and more—within a single platform.
Working with Databricks, Informatica has developed the GenAI Blueprint for Databricks Mosaic AI, which prepares enterprise data sources to be available for GenAI and RAG chain consumption.
IDMC offers data ingestion from internal and external sources, high-quality MDM, and input vectorization and storage in the first stage—alongside data preparation services provided by Databricks. Then, Informatica vectorizes semantic context, data quality metrics, data access policies, and data lineage, which is then inputted and stored into a vector database to establish a consistent layer of enterprise metadata. The blueprint then leads to defining agentic RAG executors, which involves annotating prompts with semantic metadata; prioritizing and selecting datasets based on the quality of data; enhancing and standardizing response summaries; and enforcing access controls and role-based usage restrictions.
Each of these components is unified and delivered through a variety of user experiences based on what’s most relevant for the enterprise.
Dalwadi then offered his expertise on enterprise-grade GenAI, engaging in an informative conversation with Tamm-Daniels. He emphasized the wide-spread application of GenAI, seeing application in a variety of industries—and with that expansion comes wide-spread challenges.
“It’s not that easy [to implement GenAI] today. It’s a fairly new field that’s been emerging,” said Dalwadi. “Everyone’s trying, but it’s not quite there. The primary reason…is around being able to trust their data…if you can’t trust the data that they’re built on, it’s very hard to trust the GenAI application and its outputs.”
Dalwadi further pointed out that establishing trustworthy data has much to do with data maturity.
“As these companies begin to ramp up on the data maturity curve, that’s when you’ll be able to take on more and more advantage of these advanced, high-end use cases,” he explained. “That’s why we see, when folks are starting to think about these GenAI apps, they’re combining it with data platforms like Databricks so that they can have data that they trust be imbued into these GenAI models. And as a result, these GenAI models are more trustworthy, and these are models you can actually take into production.”
This is only a snippet of DBTA’s webinar with Informatica and Databricks. For the full, in-depth webinar, featuring use cases, detailed explanations, and more, you can view an archived version of the webinar here.