Wondering what a spiking neural network is?, page-2

Great post Interloping!

I don't know if this will be of much use to everyone but here goes.

You can more or less think of all neural networks as a fitting curve. If you have ever used the trendline function in excel then you have more or less performed a very basic version of machine learning.

As an example, in a parabolic equation you have 3 constants a,b,c which can be tweaked to get the best fit. y= ax^2 + bx + c

Neural networks just take this to many, many, many constants by having an equation at each node within a layer, which receives values from the nodes from the previous layer. Because there are so many layers it would be impossible for a human to complete it, and as such they are more or less like black boxes at a node to node level, but the general principle is pretty simple.

Lets say you have a 3 layer each with 3 nodes (this would be more or less useless but good for an example), I've cut off the final 3rd layer




so nodes 1,1 1,2 and 1,3 get an input and apply a function with coefficients, in this case lets use the most basic a linear line (this is never used in truth but it makes the explanation easier) and let's track to the node 2,1 in the second layer.

1,1 calculates a_1,1x1 + b_1,1, 1,2 calculates a_1,2x1 + b_1,2, and 1,3 calculates a_1,3x1 + b_1,3

Each of these pass their calculated values on to each node in the next layer which then use these values as their various x input with their own new a and b coefficients. When someone trains a network it is more or less taking this infrastructure and optimizing all the millions of a and b values (typically by finding minima in applying chain derivatives all the way along the network) across all the nodes so that a particular set of values is equated to a particular thing, so in essence it is just curve fitting in many many many domains.

In spiking neural networks you can just think of the ax + b as a formula for the potential.

Note that the above is incredibly simplified, but it's generally how it all works.