How to Re-Invent the Cognitive Stack of Intelligence

Kevin Kelly (founding editor of Wired magazine) just wrote a near disaster of an article “The AI Cargo Cult: The Myth of Superhuman AI.” Kelly begins by attempting to tear down the assumptions of the superhuman AI hypothesis:

1. Artificial intelligence is already getting smarter than us, at an exponential rate.
2. We’ll make AIs into a general-purpose intelligence, like our own.
3. We can make human intelligence in silicon.
4. Intelligence can be expanded without limit.
5. Once we have exploding superintelligence it can solve most of our problems.

with the following arguments:

1. Intelligence is not a single dimension, so “smarter than humans” is a meaningless concept.
2. Humans do not have general-purpose minds, and neither will AIs.
3. Emulation of human thinking in other media will be constrained by cost.
4. Dimensions of intelligence are not infinite.
5. Intelligences are only one factor in progress.

You can read the article in more detail, but also make sure you read the comments. As I began to write this, I was going to refute each argument in detail. The comments to his article, should be sufficient to provide arguments against Kelly’s article. The comments are a treasure trove of ideas on what may be more important.

However, I did enjoy the article, despite many parts that I disagree with. There are, however, two good observations that I must emphasize about the nature of intelligence.

The observation that there are many kinds of intelligence and that it can’t be measured using just a single dimension. This is not a new idea. The psychologist Howard Gardner has his Theory of Multiple Intelligence where he describes 8 kinds that humans are theorized to have. It’s the same idea as Marvin Minsky’s Society of Mind. However, I like that Kelly points out that conventional computers that perform mathematical calculations or store memory are different kinds of intelligence that have long surpassed human capabilities in these areas. Nobody will doubt this. Kelly however makes the absurd argument that ‘smarter’ can’t be defined because intelligence isn’t one dimensional.

I find it important to detail Howard Gardner’s list of intelligences (refer to Wikipedia article for more detail):

Musical-rhythmic and harmonic, aka Musicality This area has to do with sensitivity to sounds, rhythms, tones, and music.

Visual-spatial, aka Spatial intelligence

This area deals with spatial judgment and the ability to visualize with the mind’s eye.

Verbal-linguistic, aka Linguistic intelligence People with high verbal-linguistic intelligence display a facility with words and languages.

Logical-mathematical, aka Reason This area has to do with logic, abstractions, reasoning, numbers and critical thinking.

Bodily-kinesthetic, aka Gross and Fine motor skils Control of one’s bodily motions and the capacity to handle objects skilfully.

Interpersonal, aka Social skills Sensitivity to others’ moods, feelings, temperaments, motivations, and their ability to cooperate in order to work as part of a group.

Intrapersonal, aka Introspection This area has to do with introspective and self-reflective capacities.

Naturalistic, aka Ecological receptiveness

This sort of ecological receptiveness is deeply rooted in a “sensitive, ethical, and holistic understanding” of the world and its complexities — including the role of humanity within the greater ecosphere.

Gardner adds a few more such as spiritual intelligence and teaching-pedagogical intelligence.

If Gardner could previously identify 8 kinds of intelligence for humans and if a machine can be shown to be better in all these 8 kinds, then of course you can show a machine to be smarter despite intelligence being multi-dimensional. Interesting enough, Brenden Lake et al. “Building Machines that Learn and Think like Humans” emphasizes a subset of Gardner’s 8 intelligences as where research should focus on.

As I write this, I realized that a good graphic may be worthwhile that shows how multiple intelligences and intuition may be related. That’s the image you see at the beginning of the post. An excellent example of the multiple intelligences coordination is shown by this recent research (must watch!):

What follows next is a prescription on how to “invent” this cognitive stack of intelligence.

The second interesting observation that Kelly makes is the idea of inventing or discovering new strategies for thinking. He writes:

Some of the hardest problems in business and science may require a two-step solution. Step one is: Invent a new mode of thought to work with our minds. Step two: Combine to solve the problem.

Inventing new ways of thinking, that is solving problems in new ways, is discovered by running simulations. Simulations are how machines ‘imagine’ alternative situations. This is how AlphaGo was able to invent new strategies. It played against itself millions of times. Kelly seems to attribute the ability to invent new ways of thinking as something that is intrinsically human. However, he fails to provide an argument or explain the origins of this “unique” capability.

I discussed in a previous article some research that’s been performed on the origins of innovation (see: Adjacent Possible). We are however still in the early stages of understanding how to build machines that can learn new thinking strategies. The only hint that seems apparent to me is the need for simulation or alternatively “game play”. Kelly provides some support for this when he writes:

We have a lot of evidence that in addition to great quantities of intelligence we need experiments, data, trial and error, weird lines of questioning, and all kinds of things beyond smartness to invent new kinds of successful minds.

It is also clear to Kelly that simulation is necessary to invent new ‘smartness’.

These two observations by Kelly hint to a very interesting approach towards developing more capable intelligent machines. This approach is not entirely new and many research groups are working on this area. Specifically, the identification that there is a need for multiple kinds of intelligence and that there needs to be way to invent new kinds of intelligence. Discovering how to do the latter is a key technology that will greatly accelerate development.

In “a roadmap to AGI”, I wrote about modular deep learning, market driven coordination and meta-learning as key capabilities for the next step. The assumptions of the former two are the need to address the decomposition and composition behavior. The latter addresses a way to learn new behavior. The latter, I’m still struggling with the details. Clearly the notion of ‘late-binding’ needs to fit somewhere and that somewhere has to do with ‘context adaptation’. My understanding of meta-learning is incomplete if I can’t understand how it relates to context adaptation. We are still in the early innings, however, it is becoming quite clear as to where interesting new research will be coming from.

One open question that I do have is whether new strategies for thinking can be learned using gradient descent as the learning method. I suspect that it cannot. New strategies are likely to be discovered by chance and the learning approach is likely something along the lines of genetic algorithms.

One thing though that I can assure you that is happening, something that surprised me about Kevin Kelly’s remark:

By exponential growth I mean that artificial intelligence doubles in power on some regular interval. Where is that evidence? Nowhere I can find. If there is none now, why do we assume it will happen soon?

Deep Learning development is accelerating at an unimaginable pace. Anyone who tries to keep up in this field knows intimately well that this indeed is happening. Unfortunately, even people like Kevin Kelly, who one would expect would be in the know, maybe completely blind to the massive developments.

BTW, this paper (https://arxiv.org/pdf/1703.10987.pdf that’s referenced by one of the commentators is bordering on the weird. Although, the arguments are much more solid than Kevin Kelly’s.