Cloudera, the only true hybrid platform for data, analytics, and AI, is unveiling Cloudera AI Inference powered by NVIDIA NIM microservices, part of the NVIDIA AI Enterprise platform. Designed to enhance the performance, security, and scalability of AI models, Cloudera AI Inference is one of the industry’s first AI inference services that provides embedded NIM microservice capability, according to the company.
Despite the rapid adoption of generative AI (GenAI) in practically every corner of business, compliance risks and governance concerns continue to rise, according to recent data from Deloitte. The hotbed of risk associated with GenAI implementation has led many organizations to transition their AI initiatives to private environments—yet this shift often creates complex, do-it-yourself processes that are unable to scale, according to Cloudera.
To combat this complexity, Cloudera AI Inference delivers secure AI development and deployment to protect against sensitive data from leaking to non-private, vendor-hosted AI model services. Cloudera AI Inference establishes a trusted data foundation—powered by NVIDIA technology—which increases performance and speed of AI-driven chatbots, virtual assistants, and agentic applications.
“Enterprises are eager to invest in GenAI, but it requires not only scalable data but also secure, compliant, and well-governed data,” said industry analyst, Sanjeev Mohan. “Productionizing AI at scale privately introduces complexity that DIY approaches struggle to address. Cloudera AI Inference bridges this gap by integrating advanced data management with NVIDIA's AI expertise, unlocking data's full potential while safeguarding it. With enterprise-grade security features like service accounts, access control, and audit, organizations can confidently protect their data and run workloads on-prem or in the cloud, deploying AI models efficiently with the necessary flexibility and governance.”
Cloudera AI Inference empowers developers to build, customize, and deploy enterprise-grade LLMs up to 36x faster with NVIDIA Tensor Core GPUs, according to the companies. Integrating UIs and APIs directly with NVIDIA NIM microservice containers, Cloudera AI Inference offers a seamless user experience that eliminates the need for command-line interfaces (CLI) and separate monitoring systems. Additionally, with Cloudera’s AI Model Registry, organizations benefit from increased security and governance with centralized access control management for model endpoints and operations.
“We are excited to collaborate with NVIDIA to bring Cloudera AI Inference to market, providing a single AI/ML platform that supports nearly all models and use cases so enterprises can both create powerful AI apps with our software and then run those performant AI apps in Cloudera as well,” said Dipto Chakravarty, chief product officer at Cloudera. “With the integration of NVIDIA AI, which facilitates smarter decision-making through advanced performance, Cloudera is innovating on behalf of its customers by building trusted AI apps with trusted data at scale.”
Other key capabilities of Cloudera AI Inference include:
- Advanced AI capabilities with NVIDIA NIM microservices that allow users to optimize open source LLMs with advanced features in natural language processing (NLP), computer vision, and more
- Hybrid cloud privacy which allows enterprises to run workloads on-prem or in the cloud with VPC deployments for improved security and compliance
- Improved scalability and monitoring via auto-scaling, high availability (HA), and real-time performance tracking
- Open APIs for model deployment, management, and monitoring for seamless integration with CI/CD pipelines and MLOps workflows
- Enterprise-level model security with service accounts, access control, lineage, and auditing capabilities
- Risk-managed deployment through A/B testing and canary rollouts for controlled model updates
To learn more about Cloudera AI Inference, please visit https://www.cloudera.com/.