BRN (ASX) TATA SNN patent

Originally posted by IndepthDiver: ↑
Another new patent using Tata convolutional spiking neural network for gesture recognition. Note that this one uses a neuromorphic event camera (the previous one used radar), so it's essentially for the same thing as the robot video demonstration they did with Brainchip. This one also references STDP:

https://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=3&f=G&l=50&d=PG01&p=1&S1=stdp&OS=stdp&RS=stdp

United States Patent Application 20210397878
1 Kind Code A1
2 George; Arun ; et al. December 23, 2021
SYSTEM AND METHOD OF GESTURE RECOGNITION USING A RESERVOIR BASED CONVOLUTIONAL SPIKING NEURAL NETWORK

Abstract
This disclosure relates to method of identifying a gesture from a plurality of gestures using a reservoir based convolutional spiking neural network. A two-dimensional spike streams is received from neuromorphic event camera as an input. The two-dimensional spike streams associated with at least one gestures from a plurality of gestures is preprocessed to obtain plurality of spike frames. The plurality of spike frames is processed by a multi layered convolutional spiking neural network to learn plurality of spatial features from the at least one gesture. A filter block is deactivated from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt. A spatio-temporal features is obtained by allowing the spike activations from CSNN layer to flow through the reservoir. The spatial feature is classified by classifier from the CSNN layer and the spatio-temporal features from the reservoir to obtain set of prioritized gestures.
Inventors: George; Arun;(Bangalore, IN); Banerjee; Dighanchal;(Kolkata, IN); Dey; Sounak;(Kolkata, IN); Mukherjee; Arijit;(Kolkata, IN)
Applicant:
Name City State Country Type

Tata Consultancy Services Limited
Mumbai

IN
Assignee: Tata Consultancy Services Limited
Mumbai
IN
Family ID: 1000005313337
Appl. No.: 17/124584
Filed: December 17, 2020

Current U.S. Class: 1/1
Current CPC Class: G06K 9/0051 20130101; G06K 9/623 20130101; G06N 3/049 20130101; G06K 9/00355 20130101
International Class: G06K 9/62 20060101 G06K009/62; G06K 9/00 20060101 G06K009/00; G06N 3/04 20060101 G06N003/04

Foreign Application Data
Date Code Application Number
Jun 18, 2020 IN 202021025784
Claims

1. A processor implemented method of identifying a gesture from a plurality of gestures using a reservoir based convolutional spiking neural network, comprising: receiving, from a neuromorphic event camera, two-dimensional spike streams as an input, wherein the two-dimensional spike streams are represented as an address event representation (AER) record; preprocessing, via one or more hardware processors, the address event representation (AER) record associated with at least one gestures from a plurality of gestures to obtain a plurality of spike frames; processing, by a multi layered convolutional spiking neural network, the plurality of spike frames to learn a plurality of spatial features from the at least one gesture, wherein each sliding convolutional window in the plurality of spike frames are connected to a neuron corresponding to a filter among plurality of filters corresponding to a filter block among plurality of filter blocks in each convolutional layer from plurality of convolutional layers; deactivating, via the one or more hardware processors, at least one filter block from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt, wherein the plurality of filter blocks are configured to concentrate a plurality of class-wise spatial features to the filter block for learning associated patterns based on a long-term lateral inhibition mechanism; obtaining, via the one or more hardware processors, spatio-temporal features by allowing the spike activations from a CSNN layer to flow through the reservoir, wherein the CSNN layer is stacked to provide at least one of: (i) a low-level spatial features, (ii) a high-level spatial features, or combination thereof; and classifying, by a classifier, the at least one of spatial feature from the CSNN layer and the spatio-temporal features from the reservoir to obtain a set of prioritized gestures.

2. The processor implemented method of claim 1, wherein the spike streams are compressed per neuronal level by accumulating spikes at a sliding window of time, to obtain a plurality of output frames with reduced time granularity.

3. The processor implemented method of claim 1, wherein a plurality of learned different spatially co-located features are distributed on the plurality of filters from the plurality of filter blocks.

4. The processor implemented method of claim 1, wherein a special node between filters of the filter block is configured to switch between different filters based on an associated decay constant to distribute learning of different spatially co-located features on the different filters.

5. The processor implemented method of claim 1, wherein a plurality of weights of a synapse between input and the CSNN layer are learned using an unsupervised two traceSTDPlearning rule upon at least one spiking activity of the input layer.

6. The processor implemented method of claim 1, wherein the reservoir comprises a sparse random cyclic connectivity which acts as a random projection of the input spikes to an expanded spatio-temporal embedding.

7. A system (100) to identify a gesture from a plurality of gestures using a reservoir based convolutional spiking neural network, comprising: a memory (102) storing instructions; one or more communication interfaces (106); and one or more hardware processors (104) coupled to the memory (102) via the one or more communication interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to: receive, from a neuromorphic event camera, two-dimensional spike streams as an input, wherein the two-dimensional spike streams are represented as an address event representation (AER) record; preprocess, the address event representation (AER) record associated with at least one gestures from a plurality of gestures to obtain a plurality of spike frames; process, by a multi layered convolutional spiking neural network, the plurality of spike frames to learn a plurality of spatial features from the at least one gesture, wherein each sliding convolutional window in the plurality of spike frames are connected to a neuron corresponding to a filter among plurality of filters corresponding to a filter block among plurality of filter blocks in each convolutional layer from plurality of convolutional layers; deactivate, at least one filter block from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt, wherein the plurality of filter blocks are configured to concentrate a plurality of class-wise spatial features to the filter block for learning associated patterns based on a long-term lateral inhibition mechanism; obtain, spatiotemporal features by allowing the spike activations from a CSNN layer to flow through the reservoir, wherein the CSNN layer is stacked to provide at least one of: (i) a low-level spatial features, (ii) a high-level spatial features, or combination thereof; and classify, by a classifier, the at least one of spatial feature from the CSNN layer and the spatiotemporal features from the reservoir to obtain a set of prioritized gestures.

8. The system (100) of claim 7, wherein the spike streams are compressed per neuronal level by accumulating spikes at a sliding window of time, to obtain a plurality of output frames with reduced time granularity.

9. The system (100) of claim 7, wherein plurality of learned different spatially co-located features are distributed on the plurality of filters from the plurality of filter blocks.

10. The system (100) of claim 7, wherein a special node between filters of the filter block is configured to switch between different filters based on an associated decay constant to distribute learning of different spatially co-located features on the different filters.

11. The system (100) of claim 7, wherein a plurality of weights of a synapse between input and the CSNN layer are learned using an unsupervised two trace STDP learning rule upon at least one spiking activity of the input layer.

12. The system (100) of claim 7, wherein the reservoir comprises a sparse random cyclic connectivity which acts as a random projection of the input spikes to an expanded spatio-temporal embedding.

13. One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors perform actions comprising: receiving, from a neuromorphic event camera, two-dimensional spike streams as an input, wherein the two-dimensional spike streams are represented as an address event representation (AER) record; preprocessing, the address event representation (AER) record associated with at least one gestures from a plurality of gestures to obtain a plurality of spike frames; processing, by a multi layered convolutional spiking neural network, the plurality of spike frames to learn a plurality of spatial features from the at least one gesture, wherein each sliding convolutional window in the plurality of spike frames are connected to a neuron corresponding to a filter among plurality of filters corresponding to a filter block among plurality of filter blocks in each convolutional layer from plurality of convolutional layers; deactivating, at least one filter block from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt, wherein the plurality of filter blocks are configured to concentrate a plurality of class-wise spatial features to the filter block for learning associated patterns based on a long-term lateral inhibition mechanism; obtaining, spatio-temporal features by allowing the spike activations from a CSNN layer to flow through the reservoir, wherein the CSNN layer is stacked to provide at least one of: (i) a low-level spatial features, (ii) a high-level spatial features, or combination thereof; and classifying, by a classifier, the at least one of spatial feature from the CSNN layer and the spatio-temporal features from the reservoir to obtain a set of prioritized gestures.
Expand
Thanks ID Any patent describing a convolutional spiking neural network can only be involving Akida since it’s the only technology that can do convolutional spiking - if the duck quacks… Merry Xmas TAHs have a great festive season, 22 will put us on the map. AIMO.

(20min delay)
Last 24.5¢		Change -0.005(2.00%)		Mkt cap ! $454.7M

Open	High	Low	Value	Volume
24.5¢	25.0¢	24.0¢	$2.351M	9.583M

No.	Vol.	Price($)
10	191799	0.245
30	1580684	0.240
23	830475	0.235
30	385456	0.230
10	189403	0.225

Price($)	Vol.	No.
0.250	372207	12
0.255	636917	22
0.260	558475	7
0.265	268537	7
0.270	469340	11


Last 24.5¢		Change -0.005 ( 2.00 %)
Open	High	Low	Volume
24.5¢	25.0¢	24.0¢	7464422
Last updated 15.58pm 03/05/2024 ?

Thanks ID :) Any patent describing a convolutional spiking...

Featured News

Macquarie profits fall 32% on 'market volatility' and low-achieving 'green investments'

Infini Resources gearing up for UAV geophys survey over Portland Creek

BPH Energy renews NT Bonaparte Basin permit

Why Princeton Professor rates BlinkLab's autism & ADHD screening app

Car EV-olution accelerates Oz into pole position

Lithium Universe ends the quarter charged up for Quebec Refinery roll-out

Dancing to that same 70s tune, China, and why I'm backing gold

Patagonia shares rise above 20% on lithium grades at maiden well in Argentina

Provident Aurum's off-market bid sends Sihayo shares soaring

Sold out signs go up for RIU Resources Roundup in Sydney

Buyers (Bids)

Sellers (Offers)

Featured News

	United States Patent Application	20210397878
1	Kind Code	A1
2	George; Arun ; et al.	December 23, 2021

Current U.S. Class:	1/1
Current CPC Class:	G06K 9/0051 20130101; G06K 9/623 20130101; G06N 3/049 20130101; G06K 9/00355 20130101
International Class:	G06K 9/62 20060101 G06K009/62; G06K 9/00 20060101 G06K009/00; G06N 3/04 20060101 G06N003/04

Thanks ID :) Any patent describing a convolutional spiking...

Featured News

Top Stories

Macquarie profits fall 32% on 'market volatility' and low-achieving 'green investments'

Infini Resources gearing up for UAV geophys survey over Portland Creek

BPH Energy renews NT Bonaparte Basin permit

Why Princeton Professor rates BlinkLab's autism & ADHD screening app

Car EV-olution accelerates Oz into pole position

Lithium Universe ends the quarter charged up for Quebec Refinery roll-out

Dancing to that same 70s tune, China, and why I'm backing gold

Patagonia shares rise above 20% on lithium grades at maiden well in Argentina

Provident Aurum's off-market bid sends Sihayo shares soaring

Sold out signs go up for RIU Resources Roundup in Sydney

Buyers (Bids)

Sellers (Offers)

Featured News