I came across this patent which I found interesting. Nothing strongly linking to Brainchip but interesting given the amount of times doorbell cameras have been mentioned by Brainchip. It mentions an AI module but one of the claims also mentions a link to a server. To me this doesn't portray a pure edge application. It also mentions unsupervised learning but only in a very vague wording.
Also note that this application was filed 18 months ago so there may be more advanced patents we can't see.
Applicant is Hanwha Techwin Co, formerly known as SamsungTechwin.
https://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=6&f=G&l=50&d=PG01&p=1&S1=%22unsupervised+learning%22&OS=%22unsupervised+learning%22&RS=%22unsupervised+learning%22
EVENT GENERATION BASED ON USER FEEDBACK BY DOORBELL CAMERA SYSTEM
AbstractA method for generating an event, which is performed by a doorbell camera system disclosed in the present disclosure may include acquiring video of a visitor, outputting the acquired video of the visitor to a user, learning the user's feedback on the visitor, storing information of the visitor and a learning result of the user's feedback on the visitor, and generating an event based on the information of the visitor and the learning result of the user's feedback.
| Inventors: | SON; Myung-hwa;(Seongnam-si, KR); KIM; Jongho;(Seongnam-si, KR); SUNG; Brian;(Seongnam-si, KR) |
|---|
| Applicant: | | Name | City | State | Country | Type |
|---|
HANWHA TECHWIN CO., LTD. |
Seongnam-si |
|
KR |
|
|
|---|
| Assignee: | HANWHA TECHWIN CO., LTD.
Seongnam-si
KR |
| Date | Code | Application Number |
|---|
| Jan 22, 2020 | KR | 10-2020-0008469 |
Claims
1. A doorbell device for generating an event for a visitor at a doorbell camera system, comprising: an input unit configured to receive first input data related to the visitor including at least one of video or audio of the visitor; a wireless communication unit configured to transmit the first input data to a user terminal and receive second input data related to a user's feedback on the first input data from the user terminal; an artificial intelligence (AI) moduleconfigured to learn the first input data based on at least one first learning algorithm to generate first output data related to the event, and learn the second input data based on at least one second learning algorithm to generate second output data related to the event; and a control unit configured to perform control to determine a danger level for the visitor based on at least one of the first output data or the second output data, and generate the event based on the danger level.
2. The doorbell device of claim 1, wherein the first output data comprises at least one of face information, body feature information, clothing information, and belongings information, of the visitor.
3. The doorbell device of claim 2, wherein in the face information of the visitor, a face region of the visitor is detected from the first input data based on a deep neural network (DNN) algorithm, and a face of the visitor is recognized from the detected face region based on a Res Net algorithm.
4. The doorbell device of claim 2, wherein the body feature information of the visitor is learned based on at least one of a Mel frequency cepstral coefficient (MFCC) algorithm, a linear predictive coding (LPC) algorithm, or a long short-term memory (LSTM) algorithm.
5. The doorbell device of claim 2, wherein the belongings information of the visitor is learned based on a you-only-look-once (Yolo) algorithm.
6. The doorbell device of claim 1, wherein the second output data comprises facial expression information of the user.
7. The doorbell device of claim 6, wherein the facial expression information of the user is generated as JSON data obtained by data-converting a human facial expression into numbers and alphabets.
8. The doorbell device of claim 1, wherein the danger level is low, medium, or high.
9. The doorbell device of claim 1, wherein the event is doorbell ignore, door unlock, emergency call connection, or non-face-to-face payment module activation.
10. A doorbell device for generating an event for a visitor at a doorbell camera system, comprising: an input unit configured to receive first input data related to the visitor including at least one of video or audio of the visitor; an artificial intelligence (AI) module configured to learn the first input data based on at least one first learning algorithm to generate first output data related to the event, and learn second input data based on at least one second learning algorithm to generate second output data related to the event; a wireless communication unit configured to receive an uplink grant from a server, transmit the first input data to the user terminal based on the uplink grant, receive a downlink grant from the server, and receive the second input data related to the user's feedback on the first input data from the user terminal based on the downlink grant; and a control unit configured to perform control to generate the event based on the first output data or the second output data.
11. The doorbell device of claim 10, wherein the first output data comprises at least one of face information, body feature information, clothing information, and belongings information, of the visitor.
12. The doorbell device of claim 10, wherein the second output data comprises at least one of facial expression information, voice information, and behavior information of the user.
13. The doorbell device of claim 10, wherein the control unit is configured to perform an initial access procedure with the server for downlink synchronization and reception of system information, and perform a random access procedure with the server for uplink synchronization.
14. The doorbell device of claim 13, wherein the initial access procedure is performed by a synchronization signal block including a first synchronization signal, a second synchronization signal, and a broadcast channel, the synchronization signal block includes four consecutive OFDM symbols, and the first synchronization signal, the broadcast channel, and the second synchronization signal are transmitted for each OFDM symbol, each of the first synchronization signal and the second synchronization signal includes one OFDM symbol and 127 subcarriers, and the broadcast channel includes three OFDM symbols and 576 subcarriers.
15. The doorbell device of claim 13, wherein the control unit is configured to perform the random access procedure by transmitting a random access preamble sequence to the server and receiving a random access response message from the server.
16. The doorbell device of claim 15, wherein the random access preamble sequence comprises a long sequence and a short sequence having different lengths, and the length of the long sequence is 839, and the length of the short sequence is 139.
17. A doorbell device for generating an event for a visitor at a doorbell camera system, comprising: an input unit configured to receive first input data related to the visitor including at least one of video or audio of the visitor; an artificial intelligence (AI) module configured to learn the first input data based on at least one first learning algorithm to generate first output data related to the event, and learn second input data based on at least one second learning algorithm to generate second output data related to the event; a wireless communication unit configured to transmit the first input data to a user terminal, receive second input data related to a user's feedback on the first input data from the user terminal, receive an uplink grant and a downlink grant from the server, transmit the first output data and the second output data to the server based on the uplink grant, and receive danger level information indicating a danger level for the visitor from the server based on the downlink grant; and a control unit configured to perform control to generate the event based on the danger level.
18. The doorbell device of claim 17, wherein the first output data comprises at least one of face information, body feature information, clothing information, and belongings information, of the visitor.
19. The doorbell device of claim 17, wherein the second output data comprises at least one of facial expression information, voice information, and behavior information of the user.
20. The doorbell device of claim 17, wherein the danger level is low, medium, or high, and the event is doorbell ignore, door unlock, emergency call connection, or non-face-to-face payment module activation.
[0183] The model learning unit 54 may learn to have a criterion for determining how the neural network model classifies predetermined data by using the acquired training data. In this case, the model learning unit 54 may train the neural network model through supervised learning using at least a portion of the training data as a criterion for determination. Alternatively, the model learning unit 54 may train the neural network model through unsupervised learning to discover a criterion by self-learning using the training data without being supervised. In addition, the model learning unit 54 may train the neural network model through reinforcement learning by using feedback on whether the result of situation determination based on the learning is correct. In addition, the model learning unit 54 may train the neural network model by using a learning algorithm including an error back-propagation method or a gradient decent method
(20min delay)