2021 BRN Discussion, page-17949

BarrelSitter

4,985 Posts.

1254

06/08/21

21:33

Post #: 55099666

Originally posted by BarrelSitter: ↑

Just some thoughts on how Akida could complement a phone camera:

Google's cloud Tensor processing unit includes a handful of Von Neumann processors (CPUs, GPUs), and DSPs. I guess that they are using a slimmed down version in Pixel 6.

https://www.engadget.com/google-tensor-faq-specs-ai-ml-160021397.html

Google’s upgraded the cameras themselves, but the promised photography improvements aren’t just from optical hardware. Tensor is behind some of it. With previous chips, the company kept running into limits when trying to improve photography on its phones. “These weren’t designed for machine learning, or AI, and certainly not designed to optimize for where Google’s headed,” he said.

...
It’s not just designed to speed up machine learning tasks on your phone, either. Osterloh said they’ve also redesigned the image signal processor, or ISP. Specifically, he said there are a “few points in the ISP where we can actually insert machine learning, which is new.”Google also reconstructed the memory architecture to make it easier to access RAM, and allow for data manipulation while processing images. There are also a few places where Osterloh said they’ve directly encoded their image processing algorithms into the hardware. He says this allows Google to do “stuff that was previously impossible to do on standard SoCs,” though he didn’t share specific details on what Tensor now enables that previous SoCs couldn’t.

So they have integrated ML into their Image signal processor (ISP), and encoded image processing algorithms into the hardware - a sort oh hybrid von Neumann/AI processor which improves image processing.

(CPU/GPU) are designed to implement the logic steps incorporated in a computer programme.

Akida SoC performs the task of data identification and classification of sensor data more efficiently than any other processor. In particular, Akida is capable of processing sensor data (cameras, microphones, LiDAR, VOC, etc.) in real time, tasks which a von Neumann processor invests massive amounts of compute power.

Akida SoC includes an ARM processor, a BRN proprietary CNN-to-SNN and data converters, and several comms interfaces. The ARM processor is used primarily for configuring the NN, but does not take part in the operation of the NN. While Akida may be able to use the processed data for limited upstream uses, it does not have the ability to make use of the processed data in a complex process. For that, it needs to operate as a co-processor with a von Neumann processor (CPU/GPU).

The Akida IP consists mainly of the NN circuit layout masks and the CNN-to-SNN and data converters masks and configuration software. It does not include the ARM Cortex processor or the standard communication interfaces. Those need to be supplied by the licensee as needed.
This gives the licensee the freedom to incorporate the Akida IP with a von Neumann processor, in a hybrid processor configuration which can be programmed to perform more complex tasks than Akida can perform, eg, ADAS/AV control in a single SoC, so that the hybrid processor can benefit from the sped and efficiency in handling the heavy duty tasks of image/voice/etc data from sensors, such that the von Neumann processor only needs to use the NN outputs as inputs to implement its programmed function.

Akida does AI & ML like nobody else.

For Google's sake, I hope they are using Akida IP.