- This patent is well worth a read as the seem to have cracked it here.
My opinion only DYOR
FF
AKIDA BALLISTA
Background - [0004]
A conventional deep convolutional neural network is comprised of many layers of neurons comprised of computed functions, whereby the input values and output value are floating-point numbers. The foremost use of convolutional neural networks is in object classification in digital images. The input values to the first layer of the neural network are samples of the signals for which classification is desired. Typical signals are sensory signals, such as visual signals, audio signals, and the like. Samples of visual signals include pixel values expressing color intensities in an image, while samples of audio signals include frequency component values as input values. Deep convolutional neural networks have three or more layers of neurons. Each layer receives inputs from the layer before it. Each input value is multiplied by a value that represents the weight of the connection, which are generally 32-bit integer or floating point numbers. Each neuron in the neural network can have many inputs, and the result of these multiplications is added to create a sum value. A non-linear function, such as the rectified linear function (ReLU) is applied to the sum value to produce an output value. The convolution is a function that is computationally applied to floating-point data to extract a feature from a defined area of the previous layer. Pooling layers are commonly inserted between convolutional layers to down-size the data. Pooling layers operate on a defined block of data from the previous layer and perform a max, average or mean pooling to reduce dimensionality.
- [0005]
Deep convolutional neural networks have been very successful in object classification tasks using image datasets. A typical deep convolutional neural network may need to perform in excess of 3 billion multiply-accumulate functions to classify a single object in an image. The processing nodes used in general-purpose computers are usually not fast enough to perform the billions of operations required for classification within a reasonable time span. Arrays of specialized multiply-accumulate devices, graphics processing units (GPU), vector processors, analog multipliers and Digital signal processors have been used to increase the throughput and reduce the latency of deep convolutional neural networks. All these devices have in common that they operate computational cores in parallel and process sequences of data rapid succession, thus being able to process large data sets in a short time. However, with great computational power comes high power consumption. A typical graphics processing unit may consume as much as 200 to 300 watts. There have been attempts to create devices that work by the same principles and consume less power, but due to their limited number of processing cores these are not capable of processing image data at the same speed. There is a need for a device that can classify objects in images at a high speed and at low power consumption.
- [0006]
Spiking neural networks have the advantage that the neural circuits consume power only when they are switching, this is, when they are producing a spike. In sparse networks, the number of spikes is designed to be minimal. The power consumption of such circuits is very low, typically thousands of times lower than the power consumed by a graphics processing unit used to perform a similar neural network function. However, up to now temporal spiking neural networks have not been able to meet the accuracy demands of image classification. Spiking neural networks comprise a network of threshold units, and spike inputs connected to weights that are additively integrated to create a value that is compared to one or more thresholds. No multiplication functions are used. Previous attempts to use spiking neural networks in classification tasks have failed because of erroneous assumptions and subsequent inefficient spike rate approximation of conventional convolutional neural networks and architectures. In spike rate coding methods, the values that are transmitted between neurons in a conventional convolutional neural network are instead approximated as spike trains, whereby the number of spikes represent a floating-point or integer value which means that no accuracy gains or sparsity benefits may be expected. Such rate-coded systems are also significantly slower than temporal-coded systems, since it takes time to process sufficient spikes to transmit a number in a rate-coded system. The present invention avoids those mistakes and returns excellent results on complex data sets and frame-based images.
SUMMARY OF THE INVENTION - [0007]
Embodiments include a system that includes a spike converter configured to generate spikes from the digital input data; and an inbound filter configured to select relevant spikes from the generated spikes. Embodiments further include a memory configured to store kernels in inverted format, and further configured to store weights indexed by channel. The embodiments also include a packet collection module configured to collect the relevant spikes until a predetermined number of relevant spikes have been collected in a packet in memory, and to organize the collected relevant spikes by channel and spatial coordinates in the packet. Finally, the embodiments include a convolution neural processor configured to perform row-by-row strides in the memory, where the convolution neural processor uses a scratchpad memory within the memory.
- [0008]
Embodiments also include a method that includes receiving digital input data, and generating, by a spike converter, spikes from the digital input data. The embodiments further include selecting, by an inbound filter, relevant spikes from the generated spikes, as well as storing kernels in a memory in inverted format, where the storing further includes storing weights indexed by channel. The embodiments also include collecting the relevant spikes until a predetermined number of relevant spikes have been collected in a packet in memory, and organizing the collected relevant spikes by channel and spatial coordinates in the packet. Finally, the embodiments further include performing, using a convolution neural processor and a scratchpad memory, a convolution in hardware using row-by-row strides in the memory.
- [0009]
Further features and advantages, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. It is noted that the specific embodiments described herein are not intended to be limiting. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein
J:
(20min delay)
