https://semiengineering.com/the-challenges-of-building-inferencing-chips/
The Challenges Of Building Inferencing chips
.....Still, when looking at chipsets or chips for AI, what’sdeveloped over the last six or eight years has been the concept of a deeplearning accelerator, the sample being the GPU, observed Roger Levinson, chiefoperating officer at BrainChip. “This is where Nvidia did brilliantly inrealizing that their floating point math processor is great for doing matrixmultiplications, which is a calculation required to do convolution on neuralnetworks. It’s an image. Convolution is an input processing thing. And that’swhat GPUs did. It has enabled a huge step forward in our capabilities in AI,and we have to be extremely thankful that we have this hardware, becausewithout it we wouldn’t have gotten anywhere. That was a technology breakthroughthat unleashed AI to be practical with the first-generation of AI, and it hasdone a great job of getting us to where we are. But the power is way too high.”
Further,the ability to do real learning is not enabled through that hardware, he said.“The traditional architecture uses a CPU or host in a data center that’s goingto be a big host, or it might be a little microcontroller, but either way theCPU is really the brains of the system. That’s what’s doing the network algorithmmanagement and running the algorithm itself. This offloads compute-heavyworkloads to an accelerator — a deep learning accelerator or a MAC acceleratoror an AI accelerator, whatever it may be called. It’s a chip that’s provisionedthrough a systolic array or some other structure in order to do very efficientmultiply-accumulates and accelerate the process of doing calculations tosupport the algorithm that is running on the CPU. The data to go in and out ofthis as it drives, the CPU says, ‘I need to run a bunch of calculations. Here’syour data, do a bunch of calculations, put it back into memory, and then I’llgo process that, and will send you the next batch.’ The whole idea is do thatas fast as possible. Folks are looking at different architectures for how tooptimize this.”
BrainChip’sapproach is to build a power-efficient neuromorphic, purpose-built processor fordoing this job. “It’s like the von Neumann computer was set up in acertain way to manage data, manipulate data and do calculations efficiently.It’s great for those types of workloads. But for AI workloads, you want aprocessor that’s different. It needs to be purpose built in order to processneural network types of information,” Levinson added.
(20min delay)