I worked for a decade at NVIDIA, as a Solution Architect to research deep learning techniques, and present solutions to customers to solve their problems and to help implement those solutions. Now, for the past 3 years I have been working on what comes next after DNNs and Deep Learning. I will cover both, showing how it is very difficult to scale DNNs to AGI, and what a better approach would be.
What we usually think of as Artificial Intelligence (AI) today, when we see human-like robots and holograms in our fiction, talking and acting like real people and having human-level or even superhuman intelligence and capabilities, is actually called Artificial General Intelligence (AGI), and it does NOT exist anywhere on earth yet.
What we actually have for AI today is much simpler and much more narrow Deep Learning (DL) that can only do some very specific tasks better than people. It has fundamental limitations that will not allow it to become AGI, so if that is our goal, we need to innovate and come up with better networks and better methods for shaping them into an artificial intelligence.
Let me write down some extremely simplistic definitions of what we do have today, and then go on to explain what they are in more detail, where they fall short, and some steps towards creating more fully capable 'AI' with new architectures.
Machine Learning - Fitting functions to data, and using the functions to group it or predict things about future data. (Sorry, greatly oversimplified)
Deep Learning - Fitting functions to data as above, where those functions are layers of nodes that are connected (densely or otherwise) to the nodes before and after them, and the parameters being fitted are the weights of those connections.
Deep Learning is what what usually gets called AI today, but is really just very elaborate pattern recognition and statistical modelling. The most common techniques / algorithms are Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Reinforcement Learning (RL).
Convolutional Neural Networks (CNNs)have a hierarchical structure (which is usually 2D for images), where an image is sampled by (trained) convolution filters into a lower resolution map that represents the value of the convolution operation at each point. In images it goes from high-res pixels, to fine features (edges, circles,….) to coarse features (noses, eyes, lips, … on faces), then to the fully connected layers that can identify what is in the image.
Recurrent Neural Networks (RNNs)work well for short sequential or time series data. Basically each 'neural' node in an RNN is kind of a memory gate, often an LSTM or Long Short Term Memory cell. RNNs are good for time sequential operations like language processing or translation, as well as signal processing, Text To Speech, Speech To Text,…and so on.
Reinforcement Learning is a third main ML method, where you train a learning agent to solve a complex problem by simply taking the best actions given a state, with the probability of taking each action at each state defined by a policy. An example is running a maze, where the position of each cell is the ‘state’, the 4 possible directions to move are the actions, and the probability of moving each direction, at each cell (state) forms the policy.
But all these methods just find a statistical fit of a simplistic model to data. DNNs find a narrow fit of outputs to inputs that does not usually extrapolate outside the training data set. Reinforcement learning finds a pattern that works for the specific problem (as we all did vs 1980s Atari games), but not beyond it. With today's ML and deep learning, the problem is there is no true perception, memory, prediction, cognition, or complex planning involved. There is no actual intelligence in today's AI.
Here is a video of how deep learning could be overtaken by methods based on more flexible spiking neural networks (flexible analog neural computers), shaped by genetic algorithms, architected into an AGI, and evolved over the next decade into a superintelligence.
We propose an AI architecture that can do all the types of tasks required - speech, vision, and other sensors that could make a much more General Artificial Intelligence.
From our AGI Patent: We specify a method for artificial general intelligence that can simulate human intelligence, implemented by taking in any form of arbitrary input data, the method comprising Learning to transform the arbitrary input data into an internal numerical format, then performing a plurality of numerical operations, the plurality of numerical operations comprises learned and neural network operations, on the arbitrary input data in the internal format, then transforming the arbitrary input data into output data having output formats using a reciprocal process learned to transform the output data from the arbitrary input data, wherein all steps being done unsupervised.
How does the brain handle vision, speech, and motor control? Well, it's not using CNNs, RNNs, nor Transformers, that's for sure. They are mere tinker toys by comparison.
First, the brain:
The cerebral cortex is a sheet 5–7 neurons (3–4mm) thick sheet folded around the exterior of the brain surface of the brain, subdivided into cortical columns of about 100,000 neurons each, which process inputs and outputs, but how?
The thalamocortical radiations are a neural structure that branches out like a bush from the thalamus at the center of the brain (the main input / output hub for the senses, vision, audio and motor outputs) with the finest end branches terminating at the cerebral cortex, feeding input/output from/to the senses to/from each of the cortical columns.
The cortical columns of the cerebral cortex are analogous to our terminal layer of autoencoders, a map storing the orthogonal basis vectors for reality and doing computations against them, including computing basis coordinates from input engrams. Our version of the thalamocortical radiations is a hierarchy of alternating autoencoders and principal component axes that we term the HAN.
Each section of the cortex is specialized for a specific type of input (visual, auditory, olfactory,… ) or output (motor, speech), and we will follow suit and have a separate HAN for each mode of input or output.
Another important structure in the brain is the hippocampus, which stores short-term memories during the day, translating sensory inputs into engrams of memory. Then, during dreaming, these memories are stored into long term memory by a combination of processes involving the hippocampus, thalamus, thalamocortical radiations, cerebral cortex, and the ROS-Inhibitory networks that handle sequential memory encoding and decoding. The hippocampus is also active in reconstructing memories, prediction, and planning, which are all different facets of the same process, irrespective of them being in the past, present or future.
We will follow nature's example to create methods and processes for computer simulations of Artificial General Intelligence (AGI) that are able to operate on general inputs and outputs that do not have to be specifically formatted, nor labelled by humans and can consist of any alpha-numerical data stream, 1D, 2D, and 3D temporal-spatial inputs, and others. The AGI is capable of doing general operations on them that emulate human intelligence, such as interpolation, extrapolation, prediction, planning, estimation, and using guessing, and intuition to solve problems with sparse data. These methods will not require specific coding, but that can rather be learned unsupervised from the data by the AGI and its internal components using spiking neural networks. Using these methods, the AGI would reduce the external data to an internal format of time-series basis coefficients.
These are time-series vectors of numbers that computers can more easily understand, be able to do math, linear algebra, supercomputing, and use databases, yet still plan, predict, estimate, and dream like a human, then be able to convert the results back to human understandable form.and have a design with a separate hierarchy of autoencoder basis set for each mode of input, to generate basis coordinates for that input / output mode, that is an artificial analogy of the thalamacortical radiations, using alternating PCA axes and autoencoders for encoding groups of engrams (encoded sensory and motor input) into a basis set for each input type, spanning all inputs that have been experienced by that input to date.
And the Encoding/decoding method that transforms real world data to/from these internal representations using a structure called the Hierarchical Autoencoder Network to divide the incoming data by features into ever more differentiated engrams till there is an orthogonal set of basis engram vectors that span the input data to date for that modality.
To manipulate sequential information In the brain, the ROS-Inhibitory network hierarchically processes sequential inputs and outputs. Generating output starts with a series of linear Rank Order Sequential neurons that fire in a sequential chain, setting a tempo or pattern with time (t), for a sequence of outputs, where that time -series ROS signal along this linear chain is the same regardless of the output to generate. This signal at each ROS neuron is then input to the root of each of a plurality of hierarchies of branching structures of inhibitory neurons that are modulated by the inhibitory signals that are modulated with the ROS signal coming from above in the hierarchy, to control which signals can get through to the next level.
In our artificial ROS-Inhibitory network, a linear series of artificial neurons fires in sequence, generating an excitatory signal when each one fires, causing each root artificial neuron in the attached branch structures to fire, and as the signal cascades down the inhibitory neural network, it is selectively inhibited by an external, time domain control signal at each neuron, by modulating the neuron’s outgoing signal by its inhibitory signal. Overall, this selects which branches of the hierarchy are activated - by controlling the inhibition at each neuron in that hierarchy.
By repeatedly training this system on a set of speech inputs, with the input to the terminal branches of the ROS-Inhibitor network reaching and training the lower levels first, then percolating upward, it would first learn a sequence of phonemes, then progressively whole words, phrases, sentences, and larger groupings, like a chorus in a song, or repeated paragraphs in legal documents.
An analogy is a music box with a set of pins placed on a revolving cylinder or disc to pluck the tuned teeth (or lamellae) of a steel comb. In the example of an adjustable music box, we can place each pin individually, or place a set of pins representing a sequence of notes that repeats often in the musical sequence. This pre-configured set of pins reduces the data needed to describe the music sequence, and makes it easier to compose music on it.
In our example, we can reduce a series of data that is often repeated to a hierarchically organized set of macros, or pre-defined sequences of that data, and not have to explicitly represent each data point.
Once trained, our system can be run forward, with the ROS / excitatory neurons firing in sequence, and playback of the trained HTSIS inhibitory signals modulating the activity of the neurons in the network to create a sequence of phonemes, words, phrases and paragraphs, reproduce video from synthetic memories, and motion control by blending hierarchical segments (directed by the AI) to generate the motion.
Now we can do processing on the hierarchical inhibitor signals as our version of memory, like training a predictor or two on them to learn conversational language, like so:
The above methods for speech would also work for controlling motion for robots and animated 3D characters, with vision, proprioception, touch, and speech as inputs, and actuator commands or animation generated as a result, using networks of predictors and solvers to plan movement and execute high level commands from speech, either from an external source, or from its own internal monologue, using language as a code to specify movement. That speech can be organized hierarchically so there are low level movements like ‘flex pinkie finger right hand 10%’ or high level commands like ‘walk forward 2 meters, turn left, and stand on one foot. Internal monologues need not be scripted, as they could be generated like our above conversation, reacting to what is happening in the world (vision, audio, touch, proprioception) and predicting what may happen next, then synthesizing intelligent movement based on training and practice.
And that is how you design a core AGI that can do speech, vision, and motion control. Each function will use similar systems derived from the core design, but will be trained and evolved to function optimally for their purpose
(20min delay)