As we all know the new CEO Sean Hehir is from HP. Somehow he found out about the revolutionary Brainchip Akida processors and decided he wanted to be a part of the company. Since Sean was still working at HP when he would have known about Brainchip, what are the odds HP is also a customer of Brainchip?
This recent patent is particularly interesting:
Pure speculation, DYOR
https://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=1&f=G&l=50&d=PG01&p=1&S1=stdp&OS=stdp&RS=stdp
| United States Patent Application | 20210357751 |
|---|
| 1 | Kind Code | A1 |
|---|
| 2 | Athreya; Madhu Sudan ; et al. | November 18, 2021 |
|---|
EVENT-BASED PROCESSING USING THE OUTPUT OF A DEEP NEURAL NETWORK
AbstractExamples forevent-based processing using the output of a deep neural network are described herein. In some examples, event format data may be provided to a spiking neural network (SNN). The SNN may perform processing on the event format data. The SNN may be trained for processing the event format data based on an output of a deep neural network (DNN) trained for processing of sensing data.
| Applicant: | | Name | City | State | Country | Type |
|---|
Hewlett-Packard Development Company, L.P. |
Spring |
TX |
US |
|
|
|---|
| Assignee: | Hewlett-Packard Development Company, L.P.
Spring
TX |
1. A method, comprising: providing an output of a deep neural network (DNN) trained for processing sensing data to a spiking neural network (SNN); performing, by the SNN, processing of event format data based on the output of the DNN; and determining a loss between the output of the DNN and an output of the SNN, wherein the DNN is disabled when the loss is within a threshold.
2. The method of claim 1, wherein the output of the DNN comprises labeled sensing data corresponding in time to the event format data.
3. The method of claim 1, further comprising synchronizing the event format data with the sensing data based on a common clock signal and a timestamp of the sensing data.
4. The method of claim 1, further comprising identifying, by the SNN, a significant event in the event format data based on the output of the DNN.
5. The method of claim 1, further comprising distinguishing, by the SNN, between a significant event and an insignificant event in the event format data based on the output of the DNN.
6. A computing device, comprising: a deep neural network (DNN) trained for image processing an image frame; a spiking neural network (SNN) to perform image processing of event format image data based on an output of the DNN; and a loss detection module to determine a loss between the output of the DNN and an output of the SNN, wherein the DNN is disabled when the loss is within a threshold.
7. The computing device of claim 6, further comprising an event processor that synchronizes the event format image data with the image frame.
8. The computing device of claim 7, wherein the event processor synchronizes the event format image data with the image frame based on a common clock signal and a timestamp of the image frame.
9. The computing device of claim 6, wherein the SNN identifies a significant event in the event format image data based on metadata included in the output of the DNN.
10. The computing device of claim 6, wherein an event capture sensor provides the event format data to the SNN based on a threshold indicating a significant change in the event format data.
11. A non-transitory machine-readable storage medium encoded with instructions executable by a processor, the machine-readable storage medium comprising: instructions to provide event format image data to a spiking neural network (SNN); and instructions to perform image processing on the event format image data by the SNN, wherein the SNN is trained for image processing the event format image data based on an output of a deep neural network (DNN) trained for image processing of image frames.
12. The machine-readable storage medium of claim 11, further comprising instructions to determine that the SNN is fully trained by the DNN based on a loss between the output of the DNN and an output of the SNN.
13. The machine-readable storage medium of claim 11, further comprising instructions to disable the DNN when the SNN is fully trained by the DNN.
14. The machine-readable storage medium of claim 11, wherein the SNN processes the event format image data without using the output of the DNN.
15. The machine-readable storage medium of claim 11, wherein the SNN is pretrained for image processing the event format image data based on the output of the DNN, and wherein the SNN is included in a computing device without the DNN.
[0019] An emerging paradigm in computing is event-driven processing, which is an aspect of research underway within the larger umbrella of brain-inspired computing, also referred to as neuromorphic computing. Event-driven processing bears a similarity to spiking and spike propagation within a human brain. Because the processing is triggered by events, the energy expended by a computing device 102 may be significantly less when compared with non-event-driven systems. For example, in a frame-based camera, an entire image frame 110 is read out periodically, even when changes between image frames 110 are minimal. For example, an entire image frame 110 may be read out and the pixels within the image frame 110 may be processed even when most of the pixels remain unchanged. In comparison, in an event-driven sensor, instead of periodically reading an image frame 110, individual pixels may be read upon detecting a change. In the context of image data, energy efficiency in cameras may be beneficial for continuous sensing at edge conditions and in cases where the cameras may be powered by a battery. For instance, cameras may be installed to monitor crops and may be powered by a battery.
[0020] While an event-driven sensor may improve efficiency on the camera capture side, a similar event-driven approach may be implemented for image processing. However, image processing pipelines and computer vision pipelines may be image frame based. For example, deep learning approaches to image processing process image frames 110 as opposed to events. A neuromorphic hardware processor may be based on the spiking paradigm. These processors may include an SNN 106. In some examples, the SNN 106 may be implemented as instructions stored in the memory that are executed by the processor.
[0036] The SNN 106 may perform processing of the event format data based on the output 114 of the DNN 104. In the context of visual data, the SNN 106 may perform image processing of the event format image data 112 based on the output 114 of the DNN 104. For example, the SNN 106 may perform facial recognition, object recognition, etc. on the event format image data 112 using the DNN output 114. As described above, the DNN output 114 may include labeled data, which the SNN 106 may use to train its image processing. In some examples, training the SNN 106 may include spike timing dependent plasticity (STDP) training or others training methods. The results of the SNN image processing are the SNN output 116.
[0058] The SNN 106 may identify a significant event in the event format image data 112 based on the DNN output 114. For example, the metadata included in the DNN output 114 may identify a certain image processing occurrence (e.g., facial recognition, object recognition). The SNN 106 may use the DNN output 114 to distinguish significant events from insignificant events in the event format image data 112. In this manner, the SNN 106 may be trained to perform image processing of the event format image data 112. In some examples, training the SNN 106 may include spike timing dependent plasticity (STDP) training or others training methods using the DNN output 114.
(20min delay)