Allowed: System and Method for Spontaneous Machine Learning and Feature Extraction, page-15

Hi kozikan,

BrainChip's CNN2SNN converter is described in

https://worldwide.espacenet.com/patent/search/family/070458523/publication/WO2020092691A1?q=WO2020092691
WO2020092691A1 AN IMPROVED SPIKING NEURAL NETWORK

[0035] While CNNs have been successful in detecting features and classifying images, they often suffer from several technological problems including high computational demands, catastrophic forgetfulness, and incorrect classification of adversarial samples. CNNs also suffer from high latency. While many-core processors and massive parallel processing can be used in CNNs to offset latency issues caused by high computational requirements, this often results in high power requirements for the CNN. For example, a CNN used to classify images in ImageNet can use as much as 2000 watts of power. This is because the CNN may have to employ a high-powered central processing unit (CPU) and one or more graphics processing units (GPUs) implemented on PeripheralComponent Interconnect Express (PCIe) add-in boards.

[0038] But conventional SNNs can suffer from several technological problems. First, conventional SNNs are unable to switch between convolution and fully connected operation. For example, a conventional SNN may be configured at design time to use a fully-connected feedforward architecture to learn features and classify data. Embodiments herein (e.g., the neuromorphic integrated circuit) solve this technological problem by combining the features of a CNN and a SNN into a spiking convolutional neural network (SCNN) that can be configured to switch between a convolution operation or a fully- connected neural network function. The SCNN may also reduce the number of synapse weights for each neuron. This can also allow the SCNN to be deeper (e.g., have more layers) than a conventional SNN with fewer synapse weights for each neuron.

[0052] In some embodiments, an input to a SCNN is derived from a video stream. TheA/D converter can convert the video stream to digital data. For example, the A/D converter can convert the video stream to pixel information in which the intensity of each pixel is expressed as a digital value. A digital camera can provide such pixel information. For example, the digital camera can provide pixel information in the form of three 8-bit values for red, green and blue pixels. The pixel information can be captured and stored in memory. The data to spike converter can convert the pixel information to spatially and temporally distributed spikes by means of sensory neurons that simulate the actions of the human visual tract.

[0089] FIG. 2A is a block diagram of a neuromorphic integrated circuit 200, according to some embodiments. Neuromorphic integrated circuit 200 can include a neuron fabric 201, a conversion complex 202, sensor interfaces 203, a processor complex 204, one or more data interfaces 205, one or more memory interfaces 206, a multi-chip expansion interface 207 that can provide a high speed chip-to-chip interface, a power management unit 213, and one or more Direct Memory Access (DMA) engines 214.




This specification also includes our favourite NPU.