Wevolver 2024STATE OFEDGE AIREPORTExploring the dynamicworld of...

Wevolver 2024STATE OFEDGE AIREPORTExploring the dynamicworld of Edge AI applications across industries


“Scaling Generative or Multi-modal AI

The Generative AI phenomenon and the related Multi-modal AI have opened a new way of services and business along with an enormous challenge of exponentially increasing costs for training and inference.Activities like ASR (Automatic Speech Recognition) through raw audio, Text-to-Speech (and vice versa), and, with the advent of Augmented Reality, more contextual scene generation are among many possibilities driving demand for compute. The challenge is that the level of sensor data, LLMs, and Large Vision Models (LVMs) is too heavy to manage on the Edge device. Therefore, it drives a much larger load at the cloud, not to mention the challenges with real-time response, security, and privacy.With the slowing down of Moore’s Law and especially Dennard’s Scaling, power and energy benefits that were taken for granted don’t provide the expected gain. Therefore, there is a great deal of architectural innovation needed to empower Edge devices.BrainChip’s DNA is that of brain-inspired, neuromorphic computing, but taking itin a digital and event-based manner rather than traditional analog. In the second generation, BrainChip introduced the Temporal Event-based Neural Networks (TENNs) that are very adept at multi-dimensional streaming data. Initially, TENNs have shown their benefit for a wide variety of streaming data solutions – consuming raw audio signals or health care data without the need for filtering to infer audio or vital signs. Similarly, they show benefits in achieving video object detection in sub-watt power envelopes. Combining the new algorithms with some innovative hardware choices– especially those that make 3D convolutions very efficient – brings a big step in efficiency—demonstrating more than 100-500x improvement in energy efficiency without compromising accuracy.In terms of disruptive potential, TENNs could revolutionize LLMs and LVMs at the Edge. Tested on prior generation transformer-based models, a TENNs-equivalent has shown that for equivalent perplexity scores (an indicator of correctness), while reducing model size and MACs/token by 3-4 orders of magnitude. More importantly, the training of these models is similar to that of CNN training and yet takes less than 1/10th the time compared to the transformer equivalent.The result is that AkidaTM with TENNs can provide radical alternatives for Edge Devices that can now handle much more complex models in a small footprint solution for vision, surveillance, hearables, automotive, healthcare, and more. Check out ourwhite paper from BrainChip”

The entire report of 100 pages is now available to download.

My opinion only DYOR
Fact Finder