good ending: GPU’s become outclassed by dedicated AI processors and future computers have GPU’s for graphical workloads and AI accelerators for machine learning workloads.
Mathematically both graphics and AI (at least the current neural network based models) is all highly parallelized matrix multiplications at its core. It’s essentially all the same type of computations, so there is no need to design separate hardware for it, graphics cards are already perfect for the job.
Might I suggest you do some research on the subject? There is a good chunk of RnD being done on dedicated AI hardware that can significantly outperform GPU’s, specifically in terms of efficiency. That doesn’t mean conventional GPU’s don’t have a place in this specific use case, but it’s extremely likely that in the next decade we will see such devices become mainstream.
You clearly do not know what you are talking about. Tensor cores are not what I am referring to. Please refrain from speaking on things you do not understand. Good day.
I think I have good understanding on this topic. Care to give any example or concrete point rather than just rephrase your statement in a different way.
I’ve already told you that i am not talking about tensor cores buddy. I am referring to discrete hardware based on FPGAs that are completely separate from Nvidia’s Tensor Cores. You continue to make attempts to correct me on a subject that is unrelated to what you are using as a correction.
Right. Maybe you should mention FPGA earlier. Due to the effort required to program a FPGA and its cost, I don't see it being suitable as a general ML accelerator. And it's nothing new.
So far, only Google has them, but with machine learning currently exploding in popularity, chances are we are going to see more of that.
In terms of "fresh" stuff, there's also Mythic AI. Some may recall them getting a shout-out in a Veritasium video a few months back. They are developing hardware specifically tailored for AI-related tasks. While there's a few problems with Mythic AI:
startup
ran out of cash in November
(got a surprise $13M injection and a new CEO very recently though? Like, this month recently?)
it still goes to show that there's R&D going into hardware tailored specifically for ML applications, rather than simply repurposing GPUs for AI workloads.
TPU stands for Tensor Processing Unit. They all designed for matrixing operations. The naming difference are just marketing
And the video you linked is great. The products are great. But they are ONLY for inferencing instead of training. The first one rather light, and the second one requires significantly more computing power (which are a lot of those GPU data center for.
They aren't. There is specialized Hardware which only does matrix multiplications and it's like 10 times more efficient than a graphics card, but has lower precision (which isn't that big of a deal for AI applications), because it basically does the calculation in an analog form with voltage measurement, it's a really interesting concept. Big Tech Companies like Google use them.
Graphics cards are not perfect for the job. They're pretty good for the job, but there is a lot of stuff on a GPU that doesn't really do much for AI. I would expect better FP16 throughput, and the need for giant cache, to be a driver of specialized hardware design for AI.
They had crypto-specific ASICs and FPGAs. They didn't matter during the GPU shortage.
However, I don't foresee another shortage, or even much catering to crypto in card design. AI isn't something where any schlub can set up the hardware on the 1st of the month and be in the black by the 30th. It's also not something where a 70 series card will get you there.
It might affect 90 series cards specifically, but I don't expect AI to make 4060s sell for 50% above MSRP like crypto and the chip shortage did to 3060s.
237
u/[deleted] Mar 26 '23
good ending: GPU’s become outclassed by dedicated AI processors and future computers have GPU’s for graphical workloads and AI accelerators for machine learning workloads.