Judging by the replies to the original post, it seems clear that nobody has seen any hard numbers for performance with the chip - despite the fact that BrainChip received the chips months ago. Now, I can understand that anyone that has been testing the chip under NDA could be prohibited from publishing what they found. But that doesn't mean that BrainChip could not provide information themselves if they wanted to.
Test problems that could be used to provide such numbers is easy to find. For example, Brainchip's developer site provides many examples of how to generate code for standard problems.
But this is just scratching the surface - ther are over 4000 such datasets that can be used for benchmarking https://paperswithcode.com/datasets in many different domains - images, text, video, audio, medical, 3D...
I've looked through all the example provided on BrainChip's website, in the hope that there would be some figures that referred not just to accuracy, but the time an AKIDA chip takes to perform the task.
But all I can see are numbers for the software simulator. For example, in the CIFAR10 example, you can see the following output Accuracy: 94.00% Akida inference on 1000 images took 23.52 s.
In the MobileNet example, you can find this Inference on 10 images took 1.24 s. Accuracy: 100.00 %
In the YOLO example, you can read car 0.2552 person 0.2852 mAP: 0.2702 Akida inference on 100 images took 22.95 s.
But my understanding is that all these performance figures were obtained with the Akida Development Environment, which is doing a software emulation of the chip. Now, I have seen BrainChip's demos where you can see a chip on a test board generating a number to say that it has effectively recognized a keyword. But those demos certainly do not allow you to say how long it took to recognize the stimulus. Was it 500 ms? Or 1 ms?? Or 10 microseconds? I have no idea. But it should be easy for BrainChip's hardware team to provide real numbers for such
I would have thought that the first thing that BrainChip's hardware team would have done after receiving the chip would have been to test how long the chip took to process 1000 CIFAR10 images (and the other problems that are illustrated in the company's documentation. It takes 23.52 seconds on the computer they were using for the ADE demo. If they could say what machine was used for that, and how long the chip took do the same task, they would instantly have some hard numbers to convince developers and investors. It is quite possible that the chip is 1000 times faster than software. But who knows?
The fact that the company has yet to provide any such information is, frankly, weird. But it's never too late....
If people at the company read Hotcopper, maybe someone could suggest that providing some actual numbers for throughput and power consumption would be much appreciated - by me at least!
And if this thread could be used for actually talking about benchmarking data, rather than discussions of other chips, then that would be very much appreciated too.
BRN Price at posting:
57.0¢ Sentiment: Hold Disclosure: Held
(20min delay)