Domino Data Lab is introducing the Domino AI Gateway, enabling companies to manage access to commercial LLMs from OpenAI, Anthropic, and more—enhancing security by holding LLM API keys.
The Domino AI Gateway can help teams access large language models securely and responsibly. It provides enterprise security to ensure that only approved users can see and use model endpoints from commercial providers.
AI Gateway runs as an internal service in Domino. Domino integrated the MLflow Deployments Server into its platform as AI Gateway's foundation.
Beyond its governance and security capabilities, Domino now incorporates LLM usage metadata into its audit trail capability. AI Gateway automatically records all model access events, further expanding enterprise compliance with security and audit policies.
Domino users also benefit from MLflow Deployments Server's API. The API lets users write their prompt code once and choose which LLM to direct it to. When users find another LLM that meets their needs, all that is necessary is to configure a new MLflow LLM endpoint.
With Domino AI Gateway, enterprises can:
- Govern LLM access throughout the GenAI application lifecycle.
- Manage user access and monitor LLM queries and associated costs.
- Enable teams to responsibly experiment with building GenAI applications using leading-edge SaaS LLM services.
- Regain control and choice over LLM providers and easily mix and match models based on your needs.
In addition to this news, Domino is announcing a new Domino Pinecone Vector Database connection to accelerate RAG application development.
The data connector enables easy, secure access to vectorized content and helps keep the knowledge repository up to date without significant effort. Connect directly to vectorized data from within the Domino workspace notebook.
Like other Domino Data Sources, the Pinecone data source offers central credential management, secure storage, audit trails, and data lineage tracking. These features expand Domino's AI governance capabilities to RAG-based generative AI applications.
Pinecone is the first vector database supported through Domino's Data Sources, with Qdrant and others following soon.
For more information about this news, visit https://domino.ai.