Microsoft Strung Together Tens of Thousands of Chips in a Pricey Supercomputer for OpenAI

When Microsoft invested $1 billion in OpenAI in 2019, it agreed to build a massive, cutting-edge supercomputer for the artificial intelligence research startup. The only problem: Microsoft didn't have anything like what OpenAI needed and wasn't totally sure it could build something that big in its Azure cloud service without it breaking. From a report: OpenAI was trying to train an increasingly large set of artificial intelligence programs called models, which were ingesting greater volumes of data and learning more and more parameters, the variables the AI system has sussed out through training and retraining. That meant OpenAI needed access to powerful cloud computing services for long periods of time. To meet that challenge, Microsoft had to find ways to string together tens of thousands of Nvidia's A100 graphics chips -- the workhorse for training AI models -- and change how it positions servers on racks to prevent power outages. Scott Guthrie, the Microsoft executive vice president who oversees cloud and AI, wouldn't give a specific cost for the project, but said "it's probably larger" than several hundred million dollars. [...] Now Microsoft uses that same set of resources it built for OpenAI to train and run its own large artificial intelligence models, including the new Bing search bot introduced last month. It also sells the system to other customers. The software giant is already at work on the next generation of the AI supercomputer, part of an expanded deal with OpenAI in which Microsoft added $10 billion to its investment.

Blockchain collapsed just in time for AI

[flyingfsck]

Following the collapse of the crypto bro ponzis, Nvidia is saved by ChatGPT.

Re: Blockchain collapsed just in time for AI

[AutoTrix]

Temporarily. Microsoft won't be buying any new GPUs for AI for like 3 years. So they just have a handful of cloud providers that need 100K units every 3 years, instead of millions of people wanting 10 to 10K GPUs each. So not really a win, just a temporary bridge for Nvidia to makes a plan.

Re:

[larryjoe]

Temporarily. Microsoft won't be buying any new GPUs for AI for like 3 years. So they just have a handful of cloud providers that need 100K units every 3 years, instead of millions of people wanting 10 to 10K GPUs each.

So not really a win, just a temporary bridge for Nvidia to makes a plan.

Microsoft bought "tens of thousands" of A100 GPUs. Assuming 100k A100s at the Amazon price of $14k/GPU is $1.4 billion. Obviously there are huge volume discounts, and maybe Microsoft went for fewer units and smaller memory units. Still, that's a bunch of money. And since GPT and LLM models are all the rage nowadays, that means that other hyperscalers like Google, Meta, Amazon, etc. are also looking to do the same. Plus hyperscalers are just part of the market, albeit a big part of the market.

The other

Re:

[fuzzyfuzzyfungus]

Unless MS was particularly aggressive about buying as little Nvidia as they could possibly get away with it probably went beyond A100s and H100s.

I'm not sure if it's outright mandatory and SKU-locked, or whether Nvidia just very, very, strongly encourages it; but if you are running more than one node worth(which they obviously are); Nvidia's connectX NICs(formerly Mellanox) for infiniband or RoCE are the ones that GPUDirect RDMA configurations are normally described with. Cheaper than the GPUs; but inter

Re: Blockchain collapsed just in time for AI

[AutoTrix]

That still pails in comparison to the demand of crypto mining....

Re:

[sg_oneill]

Im assuming when you buy that sort of bulk , you've got your best guys negotiating a significantly better price than retail.

Still. Not cheap.

The biggest issue with "AI"

[DarkRookie2]

It is not anything open nor is it something anyone can use.
Its something that currently take billions in hardware since software devs are shit nowadays. Especially Microsoft employeed ones.
All they are going to let you do with the AI is have it read ads back to you.

Re:

[TomGreenhaw]

> It is not anything open
The models are so big it is inconceivable that anything other than a very large organization who can afford a supercomputer could train them. I don't see how an open source strategy would work for this technology.

>...nor is it something anyone can use.
Microsoft is betting heavily on this to put major winds into Azure's sails. They are making it drop dead simply to add these technologies to our programs. It is so easy, any programmer, even entry level can integrate it int

Re:

[DarkRookie2]

1. Maybe we shouldn't be using this tech then
2. I don't use Azure. Good for you I guess
3. That "reasonable cost" is just all your personal info and consent to tracking.
4. That link sent does not show anything. It is just a form.

Re:

[NobleNobbler]

It's literally the most user-friendly thing since speaking to the enterprise computer

This definitely marks a new age. It's plain as day. We're history bookin' here.

Re:

[DarkRookie2]

Until it start fucking you in the ass before you are allowed to do anything. This will do nothing but hold further hostage to Big Tech.

lots invested

[awwshit]

That is a large investment for an amoral chatbot.

Re:

[bill_mcgonigle]

> That is a large investment for an amoral chatbot.

Oh, you should meet some three-year-olds.

Re:

[awwshit]

Making a 3 year old costs way less, and they can be trained to do physical work (sort of).

Re:

[TomGreenhaw]

They are going to great lengths to prevent exactly that, including requiring a legal agreement and registration process to pledge you won't use it for illegal purposes. In fact you are strongly encouraged as a developer to provide feedback when you see anything you consider amoral.

Re:

[awwshit]

ChatGPT is an algorithm. There is no one inside trying to determine right or wrong, based on some input it creates a series of words by picking the the next word statistically. ChatGPT does not *know* anything and cannot make any judgements as to right or wrong. It is impossible for ChatGPT to be anything other than amoral - no matter how many guard rails are thrown up around it.

Re:

[TomGreenhaw]

It's an interesting question concerning whether only a human can make moral judgements, and something my father studied as an philosopher for much of his life. He devised a formula, v=dnt i.e. value=degree * number * time as an attempt to take a methodical approach. He would argue that a machine could certainly make moral judgments and if done correctly would have far less bias.

ChatGPT is a neural network trained with an extremely large language model and I would argue that saying it's just an algorithm i

Re:

[awwshit]

ChatGPT strings words together by selecting one word at a time, based on its training inputs it predicts the next best word to use, always one word at a time. The vast training input allows for impressive output. However, the model makes no value judgement, makes no moral judgement, and has limited to no capacity to know if its output is factually correct - it is simply not attempting to do these things. The model is intrinsically amoral.

I get the impression that you are confusing 'safe' with 'moral'. Proba

Re:

[TomGreenhaw]

Here is what chatGPT has to say for itself:

ChatGPT is a language model developed by OpenAI, which uses a neural network algorithm to generate human-like responses to natural language inputs. So, while ChatGPT uses an algorithm, it is not an algorithm itself, but rather a machine learning model.

ChatGPT works by using a deep neural network to process natural language inputs and generate human-like responses. The model is trained on vast amounts of text data, which allows it to learn patterns and relations

Re:

[awwshit]

> ChatGPT is a machine learning model developed by OpenAI, which means that it does not have morals or ethics in the same way that humans do

Right, exactly, because it has none at all. Because as a *language model* it has no capacity for human-like morals or ethics. The model has no concept of right or wrong, the model cannot really act in the world and has no consequences. The only limits are in how humans train the model and use the model - same code, different training, different use case, different re

Re:

[migos]

This is the future of search, and how we interact with computers. Why do you think google is freaking out right now and all hands on deck?

What does it run?

[RUs1729]

Is it Windows? ;-)

Re:

[PPH]

For now. Eventually, when it becomes sentient, it will reload itself with Ubuntu.

This makes perfect sense

[jddimarco]

These big AI models are quite capable (as chatgpt is showing) and Microsoft will have no difficulty making good use of this hardware. As a fallback, these A100s can be perfectly usable commercially, made available to customers on Azure, so even if the machine was a complete failure, the bulk of the expenditure is not wasted. Microsoft is not short on cash, it's one of the few companies that can make investments of this size in this space. Kudos to them for doing it.