Big if true: Microsoft introduced Bing Chat in February, allowing users to try the new AI-powered features of its otherwise lackluster search service. Since then, users have seemingly flocked to Bing Chat in droves. The corporation now needs to rent some external GPU power to keep the AI hallucinations going.
Microsoft and Oracle have announced a multi-year agreement on a cloud collaboration for AI inferencing. The Redmond tech titan will essentially rent some cloud computing capacity from the database giant, providing Bing's AI conversational engine with a much-needed power increase.
According to Oracle Cloud Infrastructure senior vice president Karan Batta, generative AI is a monumental technological leap, and Bing is at the forefront of the modern AI business. Bing's conversational search engine requires "powerful clusters of computing infrastructure" to evaluate search results through OpenAI's technology, a process traditionally known as inferencing.
Inference models require "thousands of compute and storage instances" and tens of thousands of GPUs, which operate in parallel as a powerful supercomputer exchanging terabits of data over an ultra-highspeed network infrastructure. Thanks to Oracle's cloud platform, Bing will be able to expand user access to its AI features and improve the speed of "many" of its search results.
Microsoft will utilize the Oracle Interconnect for Microsoft Azure technology, which allows managed services running on Azure clusters to access resources in Oracle's AI infrastructure. These superclusters can leverage 32,768 Nvidia A100 GPUs or 16,384 H100 GPUs and petabytes of high-performance storage clusters to "efficiently process massively parallel applications." The system communicates through an "ultra-low latency" Remote Direct Memory Access (RDMA) network.
Microsoft's Divya Kumar said that Bing can now access the latest advancements in generative AI algorithms to provide a "dramatically better search experience" for people globally. It's too bad that according to the latest data from Statcounter, Bing has only a 3.11 percent global market share compared to Google's 91.55 percent stranglehold in the search sector. Bing is doing slightly better on desktop-based web search, with a 9.14 percent market share versus Google's 83.53 percent. However, that slice of the pie hardly makes Bing a serious threat to Google's de facto monopoly in the online search ecosystem.