Australia's largest stock trading and investment forum Australia's #1 stock forum

BRN

Click for more information

0.00%

18.5¢ brainchip holdings ltd

TATA SNN patent, page-48

First 48 Last

BarrelSitter

6,614 Posts.

2189

23/12/21

20:08

Post #: 58494792

Originally posted by IndepthDiver: ↑
Another new patent using Tata convolutional spiking neural network for gesture recognition. Note that this one uses a neuromorphic event camera (the previous one used radar), so it's essentially for the same thing as the robot video demonstration they did with Brainchip. This one also references STDP:

https://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=3&f=G&l=50&d=PG01&p=1&S1=stdp&OS=stdp&RS=stdp

United States Patent Application 20210397878
1 Kind Code A1
2 George; Arun ; et al. December 23, 2021
SYSTEM AND METHOD OF GESTURE RECOGNITION USING A RESERVOIR BASED CONVOLUTIONAL SPIKING NEURAL NETWORK

Abstract
This disclosure relates to method of identifying a gesture from a plurality of gestures using a reservoir based convolutional spiking neural network. A two-dimensional spike streams is received from neuromorphic event camera as an input. The two-dimensional spike streams associated with at least one gestures from a plurality of gestures is preprocessed to obtain plurality of spike frames. The plurality of spike frames is processed by a multi layered convolutional spiking neural network to learn plurality of spatial features from the at least one gesture. A filter block is deactivated from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt. A spatio-temporal features is obtained by allowing the spike activations from CSNN layer to flow through the reservoir. The spatial feature is classified by classifier from the CSNN layer and the spatio-temporal features from the reservoir to obtain set of prioritized gestures.
Inventors: George; Arun;(Bangalore, IN); Banerjee; Dighanchal;(Kolkata, IN); Dey; Sounak;(Kolkata, IN); Mukherjee; Arijit;(Kolkata, IN)
Applicant:
Name City State Country Type

Tata Consultancy Services Limited
Mumbai

IN
Assignee: Tata Consultancy Services Limited
Mumbai
IN
Family ID: 1000005313337
Appl. No.: 17/124584
Filed: December 17, 2020

Current U.S. Class: 1/1
Current CPC Class: G06K 9/0051 20130101; G06K 9/623 20130101; G06N 3/049 20130101; G06K 9/00355 20130101
International Class: G06K 9/62 20060101 G06K009/62; G06K 9/00 20060101 G06K009/00; G06N 3/04 20060101 G06N003/04

Foreign Application Data
Date Code Application Number
Jun 18, 2020 IN 202021025784
Claims

1. A processor implemented method of identifying a gesture from a plurality of gestures using a reservoir based convolutional spiking neural network, comprising: receiving, from a neuromorphic event camera, two-dimensional spike streams as an input, wherein the two-dimensional spike streams are represented as an address event representation (AER) record; preprocessing, via one or more hardware processors, the address event representation (AER) record associated with at least one gestures from a plurality of gestures to obtain a plurality of spike frames; processing, by a multi layered convolutional spiking neural network, the plurality of spike frames to learn a plurality of spatial features from the at least one gesture, wherein each sliding convolutional window in the plurality of spike frames are connected to a neuron corresponding to a filter among plurality of filters corresponding to a filter block among plurality of filter blocks in each convolutional layer from plurality of convolutional layers; deactivating, via the one or more hardware processors, at least one filter block from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt, wherein the plurality of filter blocks are configured to concentrate a plurality of class-wise spatial features to the filter block for learning associated patterns based on a long-term lateral inhibition mechanism; obtaining, via the one or more hardware processors, spatio-temporal features by allowing the spike activations from a CSNN layer to flow through the reservoir, wherein the CSNN layer is stacked to provide at least one of: (i) a low-level spatial features, (ii) a high-level spatial features, or combination thereof; and classifying, by a classifier, the at least one of spatial feature from the CSNN layer and the spatio-temporal features from the reservoir to obtain a set of prioritized gestures.

2. The processor implemented method of claim 1, wherein the spike streams are compressed per neuronal level by accumulating spikes at a sliding window of time, to obtain a plurality of output frames with reduced time granularity.

3. The processor implemented method of claim 1, wherein a plurality of learned different spatially co-located features are distributed on the plurality of filters from the plurality of filter blocks.

4. The processor implemented method of claim 1, wherein a special node between filters of the filter block is configured to switch between different filters based on an associated decay constant to distribute learning of different spatially co-located features on the different filters.

5. The processor implemented method of claim 1, wherein a plurality of weights of a synapse between input and the CSNN layer are learned using an unsupervised two traceSTDPlearning rule upon at least one spiking activity of the input layer.

6. The processor implemented method of claim 1, wherein the reservoir comprises a sparse random cyclic connectivity which acts as a random projection of the input spikes to an expanded spatio-temporal embedding.

7. A system (100) to identify a gesture from a plurality of gestures using a reservoir based convolutional spiking neural network, comprising: a memory (102) storing instructions; one or more communication interfaces (106); and one or more hardware processors (104) coupled to the memory (102) via the one or more communication interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to: receive, from a neuromorphic event camera, two-dimensional spike streams as an input, wherein the two-dimensional spike streams are represented as an address event representation (AER) record; preprocess, the address event representation (AER) record associated with at least one gestures from a plurality of gestures to obtain a plurality of spike frames; process, by a multi layered convolutional spiking neural network, the plurality of spike frames to learn a plurality of spatial features from the at least one gesture, wherein each sliding convolutional window in the plurality of spike frames are connected to a neuron corresponding to a filter among plurality of filters corresponding to a filter block among plurality of filter blocks in each convolutional layer from plurality of convolutional layers; deactivate, at least one filter block from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt, wherein the plurality of filter blocks are configured to concentrate a plurality of class-wise spatial features to the filter block for learning associated patterns based on a long-term lateral inhibition mechanism; obtain, spatiotemporal features by allowing the spike activations from a CSNN layer to flow through the reservoir, wherein the CSNN layer is stacked to provide at least one of: (i) a low-level spatial features, (ii) a high-level spatial features, or combination thereof; and classify, by a classifier, the at least one of spatial feature from the CSNN layer and the spatiotemporal features from the reservoir to obtain a set of prioritized gestures.

8. The system (100) of claim 7, wherein the spike streams are compressed per neuronal level by accumulating spikes at a sliding window of time, to obtain a plurality of output frames with reduced time granularity.

9. The system (100) of claim 7, wherein plurality of learned different spatially co-located features are distributed on the plurality of filters from the plurality of filter blocks.

10. The system (100) of claim 7, wherein a special node between filters of the filter block is configured to switch between different filters based on an associated decay constant to distribute learning of different spatially co-located features on the different filters.

11. The system (100) of claim 7, wherein a plurality of weights of a synapse between input and the CSNN layer are learned using an unsupervised two trace STDP learning rule upon at least one spiking activity of the input layer.

12. The system (100) of claim 7, wherein the reservoir comprises a sparse random cyclic connectivity which acts as a random projection of the input spikes to an expanded spatio-temporal embedding.

13. One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors perform actions comprising: receiving, from a neuromorphic event camera, two-dimensional spike streams as an input, wherein the two-dimensional spike streams are represented as an address event representation (AER) record; preprocessing, the address event representation (AER) record associated with at least one gestures from a plurality of gestures to obtain a plurality of spike frames; processing, by a multi layered convolutional spiking neural network, the plurality of spike frames to learn a plurality of spatial features from the at least one gesture, wherein each sliding convolutional window in the plurality of spike frames are connected to a neuron corresponding to a filter among plurality of filters corresponding to a filter block among plurality of filter blocks in each convolutional layer from plurality of convolutional layers; deactivating, at least one filter block from the plurality of filter blocks corresponds to at least one gesture which are not currently being learnt, wherein the plurality of filter blocks are configured to concentrate a plurality of class-wise spatial features to the filter block for learning associated patterns based on a long-term lateral inhibition mechanism; obtaining, spatio-temporal features by allowing the spike activations from a CSNN layer to flow through the reservoir, wherein the CSNN layer is stacked to provide at least one of: (i) a low-level spatial features, (ii) a high-level spatial features, or combination thereof; and classifying, by a classifier, the at least one of spatial feature from the CSNN layer and the spatio-temporal features from the reservoir to obtain a set of prioritized gestures.
Expand
Supersleuthing IDD,

This is fresh out of the oven - published 23 December 2021 - priority 18 June 2020 - too early to access the drawings and not yet available on Espacenet!

BRN's CSNN application (my favourite) priority 1 November 2018 - published 7 May 2020.

WO2020092691A1AN IMPROVED SPIKING NEURAL NETWORK

The dates are highly significant:

The Tata patent was first filed in India just over 5 weeks after the BrainChip patent application was published and about 7 months afer a collaboration between Brainchip and Tata was announced.

https://brainchipinc.com/brainchip-and-tata-consultancy-services-tcs-jointly-present-a-demonstration-featuring-its-akida-neuromorphic-technology-platform-at-neurips-2019/
4 December 2019
BRAINCHIP AND TATA CONSULTANCY SERVICES (TCS) JOINTLY PRESENT A DEMONSTRATION FEATURING ITS AKIDA NEUROMORPHIC TECHNOLOGY PLATFORM AT NEURIPS 2019

The paper titled “Human Gesture Recognition using Spiking Input on Akida Neuromorphic Platform” co-authored by Sounak Dey, Arijit Mukherjee from TCS Research at Tata Consultancy Services (TCS) and Gilles Bézard, Douglas McLelland from BrainChip will be presented. The demonstration will involve capturing a few hand gestures and hand positions from the audience using a Dynamic Vision Sensor camera and performing live learning and classification using the Akida neuromorphic platform. This will showcase the fast and lightweight unsupervised live learning capability of the spiking neural network (SNN) and the Akida neuromorphic chip, which takes much less data than a traditional deep neural network (DNN) counterpart, consuming much less power during training.

One year after we filed our CNN2SNN patent application, we announced a joint presentation co-authored with the Tata inventors Sounak Dey, Arijit Mukherjee .

The tata patent application takes CSNN as a fait accompli. The desrcription in the Tata specification is only about 60 paragraphs, whereas the Akida description is over 160 paragraphs. Their invention is about an application of CSNN, not about the invention of CSNN - even Nelson with his telescope to his blind eye could not miss these dots.

Can't wait till the Tata drawings are available.

BRN Price at posting: 65.0¢ Sentiment: Buy Disclosure: Held

61 Great analysis

Top Stories

Revolutionary inverter technology leads the clean energy charge

From the Floor with Federal Opposition Leader Peter Dutton at Diggers & Dealers 2024

LBT

LBT inks $4.1M deal with AstraZeneca for AI platform technology

From the Floor with Kim Beazley at Diggers and Dealers 2024

Astute sniffs 1,260 ppm lithium and thick mineralisation in Nevada

There are more pages in this discussion • 48 more messages in this thread...

First 48 Last

Add BRN (ASX) to my watchlist

(20min delay)
Last 18.5¢		Change 0.000(0.00%)		Mkt cap ! $357.5M

Open	High	Low	Value	Volume
18.5¢	18.5¢	18.0¢	$157.8K	860.3K

Buyers (Bids)

No.	Vol.	Price($)
48	1325196	18.0¢

Sellers (Offers)

Price($)	Vol.	No.
18.5¢	871667	32

View Market Depth

No.	Vol.	Price($)
48	1324896	0.180
27	1349985	0.175
38	1146416	0.170
49	916326	0.165
54	1164784	0.160

Price($)	Vol.	No.
0.185	890161	32
0.190	1223239	30
0.195	1764769	22
0.200	805013	31
0.205	831500	15

Last trade - 14.02pm 07/08/2024 (20 minute delay)

BRN (ASX) Chart