Nvidia Unveils Faster Chip Aimed at Cementing AI Dominance

Nvidia announced an updated AI processor that gives a jolt to the chip's capacity and speed, seeking to cement the company's dominance in a burgeoning market. From a report: The Grace Hopper Superchip, a combination graphics chip and processor, will get a boost from a new type of memory, Nvidia said Tuesday at the Siggraph conference in Los Angeles. The product relies on high-bandwidth memory 3, or HBM3e, which is able to access information at a blazing 5 terabytes per second. The Superchip, known as GH200, will go into production in the second quarter of 2024, Nvidia said. It's part of a new lineup of hardware and software that was announced at the event, a computer-graphics expo where Chief Executive Officer Jensen Huang is speaking.

[...] The latest Nvidia products are designed to spread generative AI -- and its underlying hardware -- to even more industries by making the technology simpler to use. A new version of the company's AI Enterprise software will ease the process of training the models, which can then generate text, images and even video based on simple prompts. The lineup also includes new chips for workstations, computers designed for heavy workloads. New AI Workbench software, meanwhile, helps users switch their work on AI models between different types of computers.

5 terabytes / s

[bill_mcgonigle]

Wow, that's /bytes/.

GDDR6x is about 20G/bits/.

Holy smokes if this checks out.

Re: 5 terabytes / s

[nosandalzone]

HBM3e memory bus width is 1024 bits versus GDDR's 32. That only works because the memory is tightly integrated into the processor core, whereas (G)DDR is from an era where memory was placed far away from the processor.

Re: 5 terabytes / s

[LiterallyHiltersADV]

You can always go faster, just by putting more stuff in parallel. The problem is that the number of interconnects between the parallel bits grows exponentially, if you want them to be causally connected (Aka what a neural net does.) If this thing has to push data back out to memory before it can go to another processorlet or whatever, that's still the bottleneck. All this is peanuts, compared to a chip that has all the memory in *registers*, in-between logic units, though. (Aka hardware neural nets. Not

Re: 5 terabytes / s

[OrangeTide]

Bus design has long been a trade off between throughput and latency. For compute you can sometimes stream operations over a bus with a lot of latency, such as PCIe. Trying to keep the bulk of the processing near the memory on either side of the bus. But things get really interesting when latency is low and you can do more complex operations or connect multiple stages and accellerators together. Such as CUDA letting you use GPU and CPU together.

Re:

[Jeremi]

Which we were promised.for.decades, btw.

The "I read about some idea once in Popular Mechanics, and now I'm angry that it hasn't happened yet" thing was funny back when Comic Book Guy did it, because it was a satire of self-entitled narcissistic nerd-dom.

It's less funny when real people unironically adopt that attitude, as if the world owes them something. It doesn't.

generative AI?

[oldgraybeard]

That is the AI that just makes it up. Right!

Re:

[thegarbz]

Everything is made up. Even your sentence there.

Re:

[93 Escort Wagon]

That is the AI that just makes it up. Right!

They prefer the term "hallucinates". It makes it sound more like the A actually has some I.

Hahaha, wait for Tesla Dojo

[haruchai]

will be a exponential step up, by polynomial orders of magnitude because no one is as smart as Elon

Re:

[darekana]

will be a exponential step up, by polynomial orders of magnitude because no one is as smart as Elon

“Frankly if they could deliver us enough GPUs, we might not need Dojo,” Musk told analysts on Wednesday during Tesla’s Q2 earnings call. “But they can’t, because they have got so many customers.”

In October, Musk said he wasn’t even sure his latest project would prove itself superior to buying Nvidia chips off the shelf—unusual given he typically demands the best from his team. Instead he was full of praise of Huang and thanked the company for prioritize some o

Re:

[Chalex]

It was same with Google, they build TPUs because they were tired of giving nvidia so much money for GPUs.

Re:

[elcor]

Do you think that 100 exa will give TSLA a higher value? I'm starting to think that wall street will wake up to the idea that raw compute = value, beyond their usual way of estimating share price.

At what price?

[LiterallyHiltersADV]

How many Apples are Nvidia product prices nowadays? Is there an equivalent to the bitcoin "value" for this condustry-du-jour, that I can look up?

The Tag

[CyberRacer]

No doubt this will be marketed in a price range to fit the budgets of all but the smallest superpower nations.