AI Chip Contenders Face Daunting 'Moats' 28
Barriers to entry in an industry dominated by TSMC and Nvidia are very high. From a report: In the drama that has just played out in Silicon Valley over the future of OpenAI, one side plot concerned an ambitious chip venture by its chief executive Sam Altman. Before he was ousted and reinstated to the helm of the company, Altman had sought to raise as much as $100bn from investors in the Middle East and SoftBank founder Masayoshi Son to build a rival to compete with sector giants Nvidia and Taiwan Semiconductor Manufacturing Co. This would be a vast undertaking. And one where $100bn may not go very far. Given that the US chip designer and Taiwanese chipmaker are critical to all things generative AI, Altman is unlikely to be the only one with hopes of taking them on. But the barriers to entry -- moats in Silicon Valley parlance -- are formidable.
Nvidia has about 95 per cent of the markets for GPU, or graphics processing units. These computer processors were originally designed for graphics but have become increasingly important in areas such as machine learning. TSMC has about 90 per cent of the world's advanced chip market. These businesses are lucrative. TSMC runs on gross margins of nearly 60 per cent, Nvidia at 74 per cent. TSMC makes $76bn in sales a year. The impressive figures make it seem as though there is plenty of room for more contenders. A global shortage of Nvidia's AI chips makes the prospect of vertical integration yet more attractive. As the number of GPUs required to develop and train advanced AI models grows rapidly, the key to profitability for AI companies lies in having stable access to GPUs.
[...] It is one thing for companies to design customised chips. But Nvidia's profitability comes not from making chips cost-efficient, but by providing a one-stop solution for a wide range of tasks and industries. For example, Nvidia's HGX H100 systems, which can go for about $300,000 each, are used to accelerate workloads for everything from financial applications to analytics. Coming up with a viable rival for the HGX H100 system, which is made up of 35,000 parts, would take much more than just designing a new chip. Nvidia has been developing GPUs for more than two decades. That head start, which includes hardware and related software libraries, is protected by thousands of patents. Even setting aside the challenges of designing a new AI chip, manufacturing is where the real challenge lies.
Nvidia has about 95 per cent of the markets for GPU, or graphics processing units. These computer processors were originally designed for graphics but have become increasingly important in areas such as machine learning. TSMC has about 90 per cent of the world's advanced chip market. These businesses are lucrative. TSMC runs on gross margins of nearly 60 per cent, Nvidia at 74 per cent. TSMC makes $76bn in sales a year. The impressive figures make it seem as though there is plenty of room for more contenders. A global shortage of Nvidia's AI chips makes the prospect of vertical integration yet more attractive. As the number of GPUs required to develop and train advanced AI models grows rapidly, the key to profitability for AI companies lies in having stable access to GPUs.
[...] It is one thing for companies to design customised chips. But Nvidia's profitability comes not from making chips cost-efficient, but by providing a one-stop solution for a wide range of tasks and industries. For example, Nvidia's HGX H100 systems, which can go for about $300,000 each, are used to accelerate workloads for everything from financial applications to analytics. Coming up with a viable rival for the HGX H100 system, which is made up of 35,000 parts, would take much more than just designing a new chip. Nvidia has been developing GPUs for more than two decades. That head start, which includes hardware and related software libraries, is protected by thousands of patents. Even setting aside the challenges of designing a new AI chip, manufacturing is where the real challenge lies.
This is how it was always going to be (Score:5, Insightful)
No one small is going to make it big. M$ is not going to allow that.
Re:This is how it was always going to be (Score:4, Interesting)
they will do everything in thier power to keep everyone else out.
Meta has already open-sourced powerful tools, including LLaMA, so your conspiracy theory is wrong.
M$ is not going to allow that.
M$ partnered with Meta on LLaMA development so that part of your conspiracy theory is wrong too.
Re: (Score:2)
While almost anyone can use a trained AI for minimal investment, doing the training still requires enormous amounts of money, and, more important, the major players are the ones pushing for legal restrictions. Keep a close eye on the regulations being proposed; they will all have the effect of making smaller operators impossible without the active cooperation of the major players (at at cost).
Re: (Score:2)
You can train all sorts of useful models on a regular old desktop. You can train a decent number of useful things on a laptop CPU. It's not going to be a chatGPT rival, but special purpose language models are often more useful anyway, and there are way more things you can do than chat.
Re: (Score:1)
Re: (Score:3)
LLaMA is an LLM model. Models run on hardware. The article is about difficulty of challenging existing hardware vendors.
You can run AI models on any hardware, from standard CPUs, to GPUs. But this article is focused on specialized AI hardware such as TPU (Google), TSP (Groq), Tranium (Amazon), or AIU (IBM).
TL;DR - hardware is hard.
Designing a new chip is hard. Tape-out of a new design can be incredibly expensive ($50-$100m). Integrating new chips into boards and systems is also incredibly expensive. Bu
Re: (Score:2)
they will do everything in thier power to keep everyone else out.
Meta has already open-sourced powerful tools, including LLaMA, so your conspiracy theory is wrong.
M$ is not going to allow that.
M$ partnered with Meta on LLaMA development so that part of your conspiracy theory is wrong too.
The reason Microsoft is willing to be such a good neighbor is that the free versions of LLaMA, ChatGPT, etc. are mostly public curiosities. They don't generate income because they're not yet good enough. They do generate buzz, which might be useful for the future products that will be more useful, won't be free in terms of money, and won't be as free in terms of availability.
TSMC is a problem (Score:5, Insightful)
While it's true that they make quality products, depending on a single supplier is insane, especially when the supplier is on disputet territory
Regardless of cost and difficulty, we need alternate suppliers
Re: (Score:2)
One of my very first jobs, we couldn't use a desirable chip because there was no second source.
Re:TSMC is a problem (Score:5, Informative)
First, the journalist who wrote TFA is a moron. Sam Altman wants to compete with Nvidia, not TSMC. He wants to design chips, not fab them.
2nd, geographical dispersion is important, but not necessarily by two different companies. TSMC is building a fab in Arizona, which solves many of the geography issues.
Re: (Score:3)
TSMC is building a fab in Arizona, which solves many of the geography issues.
Until there's not enough water in the desert for a fab plant, which uses quite a lot of water. Arizona is headed for a major water crisis, and it's not too far off.
Re: (Score:3)
As long as they can prioritize the water usage (namely make sure the golf courses and cotton farming aren't affected, of course) it should be fine.
But wouldn't a billion+ dollar chip fab plant be able to recycle the water they use though? it's not as if the water completely disappears from the earth
Re:TSMC is a problem (Score:4, Informative)
There isn't really any sudden looming crisis of fabs shutting down. As water gets more scarce, you work on using less and re-using more.
Re: (Score:2)
In a capitalist system a monopoly is often the predictable outcome. Market power begats more market power until only one company remains.
Regulations try and slow and control this, but as we have seen since the 80's with various deregulation pushes you end up with regulatory capture, or companies so rich that buying favorable legislation/legislators become cheaper than innovating.
So do we break up Nvidia? Do we mandate certain market shares maximums and end up with two slow moving duopolies vaguely creatin
Re: (Score:3)
a good start might be a thorough review of patents and the patent process, it's having the exact opposite of it's intended effect..
They exist to foster the sharing of information, after allowing the inventing a company a reasonable head start (fine)
but currently they're being used to thwart competition, and the lifespan of the patent far exceeds the timeframe a given technology is relevant. Also granting them for dubious/obvious things doesn't help the situation.
Re: (Score:2)
IBM was well aware of that fact when making their PC. Intel only got the job because they could provide a second supplier named AMD.
Limit patents (Score:1)
> is protected by thousands of patents.
I believe there should be an upper limit on the percent of royalties required to be payed out. Otherwise, oligopolies just get oligoplier, hurting competition.
A good many patents are too damned trivial or obvious anyhow.
Re: (Score:2)
I tend to agree. 20 years is much too long for protection, and it is easy to extend for a while beyond that with trivial changes. A step in the right direction would be to limit patents to 5 years with a hard stop.
Re: (Score:2)
>20 years is much too long...limit patents to 5 years with a hard stop.
This fixes one problem while creating another.
Imagine you are poor and patent an incredibly awesome new innovation. You need to spend 5 years raising capital, hiring employees, building infrastructure needed to mass produce it... Then just before you start selling, big monster established company XYZ jumps in and captures the market because your patent is now expired. Your patent even provided them a roadmap to copy you and crush you
Re: (Score:2)
Patents by definition make your invention public. By your own argument, you'd be better off keeping your invention private while you work on it. Need help? Non-Disclosure Agreements are a thing. The point of a patent is that it 'teaches' someone of average skill in the art how to do something new. Want protection? Then don't teach your competitors how to do the thing.
Re: (Score:2)
Patents should have an exponential function.
First patent is free.
You thousandth patent is $10Million per patent.
And no cross licensing from related entities.
Not iillogical at all. (Score:3)
It looks like a duopoly is in control of a very important industry.
He might have not replaced any of those with $100bn but he would definitely get in the game and start getting traction.
Like the Chinese with their rumored chips are doing.
Re: (Score:2)
Intel is in the race. It's just damnably hard to catch up the incumbents who have decades of experience in the field, especially when it comes to software.
Start small (Score:2)
Help AMD (Score:3)
They'd be better off giving AMD some money to write decent libraries and add support to PyTorch and Tensorflow, or doing it for them. A hundred million dollars there should establish an Nvidia competitor much more reliably than a hundred billion dollars trying to become the next Transmeta.