Nvidia's Flagship AI Chip Reportedly 4.5x Faster Than Previous Champ

Nvidia announced yesterday that its upcoming H100 "Hopper" Tensor Core GPU set new performance records during its debut in the industry-standard MLPerf benchmarks, delivering results up to 4.5 times faster than the A100, which is currently Nvidia's fastest production AI chip. Ars Technica reports: The MPerf benchmarks (technically called "MLPerfTM Inference 2.1") measure "inference" workloads, which demonstrate how well a chip can apply a previously trained machine learning model to new data. A group of industry firms known as the MLCommons developed the MLPerf benchmarks in 2018 to deliver a standardized metric for conveying machine learning performance to potential customers. In particular, the H100 did well in the BERT-Large benchmark, which measures natural language-processing performance using the BERT model developed by Google. Nvidia credits this particular result to the Hopper architecture's Transformer Engine, which specifically accelerates training transformer models. This means that the H100 could accelerate future natural language models similar to OpenAI's GPT-3, which can compose written works in many different styles and hold conversational chats. The chip, which is still in development, is forecasted to replace the A100 as the company's flagship data center GPU. The U.S. government imposed restrictions last week on exports of the chips to China, leading many to fear that Nvidia might not be able to deliver the H100 by the end of the year. However, Nvidia clarified that the U.S. government will allow it to continue development of the chip in China, "so the project appears back on track for now," reports Ars.

Hmm.

[CaptainLugnuts]

Knowing nVidia, it only draws 8x the power.

Re:

[backslashdot]

Good point, though unlikely. If it does use more than 4.5x the power then it's not superior to the A100 and is pretty useless.

Re:

[Moblaster]

Skynet will use any power it can get in its quest for self-awareness.

Re: Hmm.

[Fons_de_spons]

Then it will first have to beat Putin.
Wait...
That is probably how everything started!

Re:

[bloodhawk]

Silly statement, power efficiency is often NOT the primary concern, more raw performance even if less efficient is often highly desirable, especially in an area like this were processing power is the bottleneck.

Re:

[backslashdot]

If that was the case, all you need is more of the older processors.

Useful only for advertising

[BrendaEM]

Meh

Super fast and grand pattern matching.

[Fly Swatter]

And they call this 'AI'.

-yep I'm one of those - deal with it.

GPUs are now AI chips?

[splutty]

I guess they need a new name to sell new stuff.. But these are no different than any normal GPU pipeline, except in scale.

No graphics / Re:GPUs are now AI chips?

[nniillss]

NVIDIA A100 cards do not have video outputs (like HDMI) and, I think, also no raytracing units. So they are definitely not video cards and not really (or mostly) GPUs.

Re:

[Anonymous Coward]

They are different as they don't have video out. They are designed purely for processing and if you bought one as a video card you would be sorely disappointed.

Misleading title as per usual

[illogicalpremise]

It is not "4.5x faster".

It achieves that in only a limited application of BERT transformer due to specific optimisations for that. In all the other tasks it appears to be less than twice as fast as the previous chip they are comparing it to. An improvement to be sure but not nearly as much as the headline implies unless your specific application uses BERT.