Trothlis said: As it currently stands, I am unfortunately more...

Originallypostedbytrothlis

Iwouldliketohavedoneabitmoreresearchbeforepostingagainhere,butIwantedtoprovideanupdatebeforetheAGMsothatyouguyscanusethistoinformanyquestionsyoumaywanttoraise.

InearlyAprilIemailedKrisCarlsonasanalternatepathtotryandresolvemyconcernsabouttheAkidahardwareresources.AfewdayslaterIreceivedanemailfromPetervanderMade.Thisstartedalengthyseriesofemailsexchangedoverthelast7weekstotryandsortoutmyconcernsregardingthehardwareresourcesthatarerequiredtomeettheclaimspublishedforAkida.Achronological,26pagecopyoftheentireexchangeisattachedtothispost(PVDM_Correspondence.pdf),andwasonlyredactedtoremovecontactinformation.IntheemailsfromMay13thand15thhegavepermissiontosharetheinformation.

Someofthehighlightsfromthisdiscussion:

1. TheAkidaneuralarchitectureisafairlysignificantdeparturefromSNAP64andthepublishedpatents,andemployssomekindofhardwarethatcanbereallocatedbetweensynapsesandneurons.
2. Theadvertised10Bsynapsenumberis"equivalentsynapses",whichiseasilycalculatedfromthenetworkdefinitionbysummingthe(filter_size*number_of_strides)*filter_countacrossalllayers.Incontrast,thenumberofphysicalsynapsesinanAkidaimplementationisthesumoffilter_size*filter_countacrossalllayers.
3. Itisstillambiguouswhatthe1.2Mneuronnumberrefersto,butmostrecentlyitwasdefinedasanaverageneuroncount.Seethe2ndconcernbelow.
4. Peterwasconsistentinstatingthatthehardwareisnotmultiplexed.IftheAkidaspecificationislegitimate,thenIthinkthataproperinterpretationofthisisprobablycentraltotheexplanation.
5. PeterhasbeenfarmoreavailableandresponsivethanIanticipated.Hewasawareofthisthreadandexpressedadesiretoresolvetheconcerns.HewillprobablythenalsobeawareofthispostandIhopeitspursfurtherclarification.

Andafewkeyconcerns:

1. Afterestablishingthetechniqueforcalculatingthenumberofhardwareentities,thenetworkwhichwasgivenastheexemplarfor10billionequivalentsynapsesappearstorequireresourcesthatarenotfeasiblefora$10-$20chipwitha28nmprocess(andprobablynotevenforanypriceortechnology).Icouldbewrong,butsomethinginthedescriptionisgoingtohavetochangeformetoseethat.
2. Itsometimesseemedthattheexplanationswerechanginginresponsetomyfurtherqueries.Forexample,inthefirstemailonApril5Peterstatedthatthepublishedneuronandsynapsecountswerethe"maximumnumberofneuronsandmaximumnumberofsynapses".Thiswasrestatedas1.2Mphysicalneuronsand10B"equivalentsynapses"laterthesameday.ButthenintheMay17emailhesaysthat"The1.2Mneuronswepublishedisanaveragefigure"afterIobservedthatthegivennetworkrequiredmorethan1.2Mphysicalneurons.
3. InthecourseofthediscussionabouttheimageclassifierCNNs,PetersuggestedthatthediscrepancywasduetoCNNsreporting"neuronequivalents"anditwasinferredthattheAkidaphysicalneuronwasreusedoneachconvolutionalstride.However,thatdidnotalignwiththesubsequentdescriptionoftheAkidaphysicalneuronandthetechniquegivenforcalculatingphysicalneuronsinAkidaendedupbeingidenticaltothethecountofneuronsinaCNN.

Asitcurrentlystands,IamunfortunatelymoreconcernedabouttheviabilityofAkidathanIwasatthestartofthisinquiry.Despitehavingclarifiedmyunderstandingofthedesignandpublishednumbers,thetaskofextractingtheseclarificationswasconfusingandthetechnologystillappearstometonotbefeasibleasitisdescribed.Iamopentothepossibilitythattherearestilladditionalexplanationsthatwouldresolvethis-butthereisgoodreasonwhyLoihihas"only"131Kneuronsand130Msynapses.Iamcurrentlyawaitingaccesstothedevelopmentenvironmenttoseeifthatcanprovideadditionalclarification.IfIhadtoguessatanexplanationwhichwouldchangemymind,IthinkthatthechipspecificationscouldbefeasibleifthereallocatablehardwareisjustspaceinSRAM,andthesynapticandneuralhardwareentitiesareactuallybeingreusedwithinalayer(i.e.,multiplexing,orserialization,orpipelining)bysequentiallypullingandstoringvaluestotheSRAM.ButPeterrepeatedlystatedthatthisisnotthecase.It'spossiblethatthereisaseveredisconnectbetweenusonwhatthatactuallymeans.

Lastly,IalsocontactedSocionexttoconfirmthattheMoUwasinplaceandaskedwhethermaterialshadbeenprovidedbyBrainchipfortheirevaluation.TheyrefusedtosayanythingandwouldonlyrefermetotheBrainchippressrelease.Idon'tthinkthere'smuchtoreadintothat,butIthoughtIwouldshare.

IfanybodyhasanyquestionsthattheywanttoaskmebeforetheAGM,youshoulddosobeforethe24th.Myavailabilitywillbelimitedthereafter.

Trothlis said: As it currently stands, I am unfortunately more concerned about the viability of Akida than I was at the start of this inquiry. Despite having clarified my understanding of the design and published numbers, the task of extracting these clarifications was confusing and the technology still appears to me to not be feasible as it is described. I am open to the possibility that there are still additional explanations that would resolve this - but there is good reason why Loihi has "only" 131K neurons and 130M synapses.

Hi Trothlis ,all that research only to end up as you started still concerned and doubting Akida's 'viability' ,you must find it frustrating that even with your understanding of the design and published numbers of Akida that you found Peters assistance confusing.
Trothlis, some of the Akida design/explanations you seek maybe too hot !! …….. to be honest I would be more concerned if you figured out how Akida works with all those MEGA neurons and synapse , poor little cousin Loihi.
Trothlis thanks for efforts, but at the end of the day the only Akida opinion that will truly matter is the one Socionext will make ! …...anybody else sweating bullets