Meta and Microsoft To Buy AMD's New AI Chip As Alternative To Nvidia's (cnbc.com) 16
Meta, OpenAI, and Microsoft said at an AMD investor event today that they will use AMD's newest AI chip, the Instinct MI300X, as an alternative to Nvidia's expensive graphic processors. "If AMD's latest high-end chip is good enough for the technology companies and cloud service providers building and serving AI models when it starts shipping early next year, it could lower costs for developing AI models and put competitive pressure on Nvidia's surging AI chip sales growth," reports CNBC. From the report: "All of the interest is in big iron and big GPUs for the cloud," AMD CEO Lisa Su said Wednesday. AMD says the MI300X is based on a new architecture, which often leads to significant performance gains. Its most distinctive feature is that it has 192GB of a cutting-edge, high-performance type of memory known as HBM3, which transfers data faster and can fit larger AI models. Su directly compared the MI300X and the systems built with it to Nvidia's main AI GPU, the H100. "What this performance does is it just directly translates into a better user experience," Su said. "When you ask a model something, you'd like it to come back faster, especially as responses get more complicated."
The main question facing AMD is whether companies that have been building on Nvidia will invest the time and money to add another GPU supplier. "It takes work to adopt AMD," Su said. AMD on Wednesday told investors and partners that it had improved its software suite called ROCm to compete with Nvidia's industry standard CUDA software, addressing a key shortcoming that had been one of the primary reasons AI developers currently prefer Nvidia. Price will also be important. AMD didn't reveal pricing for the MI300X on Wednesday, but Nvidia's can cost around $40,000 for one chip, and Su told reporters that AMD's chip would have to cost less to purchase and operate than Nvidia's in order to persuade customers to buy it.
On Wednesday, AMD said it had already signed up some of the companies most hungry for GPUs to use the chip. Meta and Microsoft were the two largest purchasers of Nvidia H100 GPUs in 2023, according to a recent report from research firm Omidia. Meta said it will use MI300X GPUs for AI inference workloads such as processing AI stickers, image editing, and operating its assistant. Microsoft's CTO, Kevin Scott, said the company would offer access to MI300X chips through its Azure web service. Oracle's cloud will also use the chips. OpenAI said it would support AMD GPUs in one of its software products, called Triton, which isn't a big large language model like GPT but is used in AI research to access chip features.
The main question facing AMD is whether companies that have been building on Nvidia will invest the time and money to add another GPU supplier. "It takes work to adopt AMD," Su said. AMD on Wednesday told investors and partners that it had improved its software suite called ROCm to compete with Nvidia's industry standard CUDA software, addressing a key shortcoming that had been one of the primary reasons AI developers currently prefer Nvidia. Price will also be important. AMD didn't reveal pricing for the MI300X on Wednesday, but Nvidia's can cost around $40,000 for one chip, and Su told reporters that AMD's chip would have to cost less to purchase and operate than Nvidia's in order to persuade customers to buy it.
On Wednesday, AMD said it had already signed up some of the companies most hungry for GPUs to use the chip. Meta and Microsoft were the two largest purchasers of Nvidia H100 GPUs in 2023, according to a recent report from research firm Omidia. Meta said it will use MI300X GPUs for AI inference workloads such as processing AI stickers, image editing, and operating its assistant. Microsoft's CTO, Kevin Scott, said the company would offer access to MI300X chips through its Azure web service. Oracle's cloud will also use the chips. OpenAI said it would support AMD GPUs in one of its software products, called Triton, which isn't a big large language model like GPT but is used in AI research to access chip features.
AMD's weak point (Score:5, Insightful)
AMD's problem throughout its existence has been software side. Both drivers and compatibility with existing third party software.
But both Meta and Microsoft have a veritable army of software engineers to work on this issue, so they are actually the kind of a partner that AMD would need to compete with nvidia's supremacy in this field. Going to be interesting to see how this pans out.
Re: (Score:3)
Going to be interesting to see how this pans out.
Only beneficial, I suspect.
NV's near-fucking-monopoly on ML training isn't doing anyone other than NV any good.
Re: (Score:2)
The question is whether their work will pay dividends on real operating systems, or only really benefit Windows. With today's Microsoft I could see it going either way, but obviously I know which way they want it to go.
Re: (Score:2)
Meta doesn't run on Windows. And Microsoft has moved a lot of server infrastructure to Linux.
This isn't Microsoft of the early 2000s, when most of their income came from Windows unit. Nowadays, their biggest source of income is Azure and related cloud services.
Um...don't? (Score:1)
'cuz it's Snake Oil?
The real question is how much (Score:2)
That large customers seeking a second source to Nvidia would publicly announce buying AMD is not a surprise. This is not a new thing. These companies have been buying AMD all along. The only important question is how much -- how many units, how much money.
It's also interesting to note that neither Microsoft nor Meta mention training. Nvidia is much more dominant in training, with anywhere from 85-95% of the market. It would be in training where the beefy MI300 would seem be targeted. The large data ce
Patents? (Score:1)
Nvidia has literally thousands of AI related patents. How long before we start seeing violation claims?
Re: (Score:2)
I doubt that Microsoft has many patents on graphics, after all it's not in any of their core areas of competence. If they had anything that they could use against Nvidia they would have been beating NV over the head with them for years. MS buys their graphics from other companies.
And MS + AMD haven't been working on AI long enough to have any major patents in that area. NV has been working on AI for almost 20 years and still has the majority of AI hardware expertise.
Remember that Microsoft depends on (and w
When will they support Linux and MDEV? (Score:3)
The problem with all of AMDs offerings in this space is that they simply donâ(TM)t offer anything for smaller deployments.
Nobody except AMD supports ROCm, their documentation is piss poor and lagging behind on features, MxGPU for Linux hasnâ(TM)t been updated since 2019.
The M300X is basically 8 GPUs bolted together, so 24GB/GPU, that is pretty small for modern AI models. If this behemoth is supposed to compete with an H100 it looks like AMD should stick to gaming.
Re: (Score:2)
The problem with all of AMDs offerings in this space is that they simply donÃ(TM)t offer anything for smaller deployments.
Basically yeah. And they don't realize that "deployments" go all the way down. I can grab any old rando NVidia card from wherever, do a "pip install pytorch" and go to town. This works from anything such as a silly little GTX1650 or whatever the free tier on Colab is up to the H100.
Sure if you want a newer feature (and those are never needed ot get started) you need a card that supp
In a gold rush, sell the shovels (Score:2)