New Relic, the all-in-one observability platform for every engineer, is introducing New Relic AI monitoring (AIM), an APM solution for AI-powered applications that provides engineers unprecedented visibility and insights across the AI application stack, making it easier to troubleshoot and optimize their AI applications for performance, quality, cost, and responsible use of AI.
With more than 50 integrations and features like LLM response tracing and model comparison, AIM helps teams build and run LLM-based applications with confidence.
AIM provides end-to-end observability to any AI-powered applications. Engineers can access a single view to troubleshoot, compare, and optimize different LLM prompts and responses for performance, cost, security, and quality issues including hallucinations, bias, toxicity, and fairness.
It provides engineers with full visibility on all components of the AI stack alongside services and infrastructure so that they have the data they need for the responsible use of AI and to prove their compliance with forthcoming AI regulations.
“With every organization integrating AI into their products and processes, AI workloads are now part of modern organizations’ application architectures,” New Relic chief product officer Manav Khurana. “With AI monitoring, we have applied our deep expertise from inventing cloud APM to providing end-to-end visibility into AI-powered applications to help businesses manage performance, costs, and the responsible use of AI.”
Key features and use cases include:
- Auto instrumentation: New Relic agents come equipped with all AIM capabilities, including full AI stack visibility, response tracing, model comparison, and more for quick and easy setup.
- Full AI stack visibility: Holistic view across the application, infrastructure, and the AI layer, including AI metrics like response quality and tokens alongside APM golden signals.
- Deep trace insights for every LLM response: Trace the lifecycle of complex LLM responses built with tools like LangChain to fix performance issues and quality problems such as bias, toxicity, and hallucination.
- Compare performance and costs: Track usage, performance, quality, and cost across all models in a single view; optimize use with insights on frequently asked prompts, chain of thought, and prompt templates and caches.
- Enable responsible use of AI: Ensure safe and responsible AI use by verifying that responses are appropriately tagged to indicate AI-generated and are free from bias, toxicity, and hallucinations using response trace insights.
- Instantly monitor your AI ecosystem: The most comprehensive solution for monitoring the entire stack of any AI ecosystem with more than 50 integrations and quickstarts including:
- Orchestration framework: LangChain
- LLM: OpenAI, PaLM2, HuggingFace
- Machine learning libraries: Pytorch, TensorFlow
- Model serving: Amazon SageMaker, AzureML
- Vector databases: Pinecone, Weaviate, Milvus, FAISS
- AI infrastructure: Azure, AWS, GCP
AIM is now available in early access to New Relic users across the globe. Users can sign up to request early access, which is included as a part of New Relic’s simplified, consumption-based pricing.
For more information about this news, visit www.newrelic.com.