IBM is announcing that Mixtral-8x7B—the popular, open source large language model (LLM) developed by Mistral AI—is available on the watsonx AI and Data platform. Now offering an enhanced version of Mixtral-8x7B, this latest delivery of the LLM echoes IBM’s commitment to provide the foundational models for its clients to innovate, according to the company.
Mixtral-8x7B is a formative mixture of Sparse modeling—a process that detects and uses only the most relevant parts of data—and a “Mixture-of-Experts” technique—which combines different models (aka, “experts”) that specialize in and solve different parts of a problem, according to IBM. This combination of technology allows Mixtral-8x7B to quickly process and analyze high quantities of data, generating context-specific insights.
With Mixtral-8x7B’s newfound availability on the watsonx AI and Data platform, IBM has applied quantization—a process which reduces model size and memory requirements for LLMs—increasing the model’s throughput by 50% when compared to the original offering. This can potentially reduce latency by 35-75%, depending on batch size, and radically boost time-to-insights.
IBM’s optimization of Mixtral-8x7B can yield accelerated processing, lowered costs, reduced energy consumption, and the choice and flexibility to scale AI solutions across the business, according to the company. This continues to expand IBM’s watsonx.ai model catalog, offering its clients the latest and greatest capabilities, languages, and modalities.
“Clients are asking for choice and the flexibility to deploy models that best suit their unique use cases and business requirements,” said Kareem Yusuf, Ph.D, senior vice president, product management and growth, IBM Software. “By offering Mixtral-8x7B and other models on watsonx, we’re not only giving them optionality in how they deploy AI—we're empowering a robust ecosystem of AI builders and business leaders with tools and technologies to drive innovation across diverse industries and domains.”
To learn more about Mixtral-8x7B, please visit https://www.ibm.com/us-en or https://mistral.ai/.