An interview with Monica Anderson — Part 2

Artificial General Intelligence (AGI) is an emerging field aiming at the building of “thinking machines”; that is, general-purpose systems with intelligence comparable to that of the human mind. What is currently labeled ‘artificial intelligence’ is largely narrow automated knowledge work, lacking the flexibility and adaptability seen in animal intelligence. The pursuit of AGI begins at a foundational level, asking fundamental questions about models of cognition, knowledge acquisition, making choices through reason, thinking and conceiving the world in adaptive and intuitive ways.

Part 2 of an interview with Monica Anderson, an AGI researcher from California.

Read part 1 here.

___________________________________

You emphasize the importance and value of “artificial understanding” of human language. What are the current “natural language processing” systems (Siri, Alexa, chat-bots, etc.) doing and how does this differ from what AGI is striving for w/regards to working with language?

None of the language understanding systems go beyond identifying words correctly in context; this is a major step forward, but not enough. Some language tasks, such as disambiguation and generation, require knowledge outside of the language itself — they require knowledge of the world. They will get some of it by just reading text. The jury is out about whether this is enough, but the last few years has seen a lot of people switch to a position close to what I believe: Yes, we can learn close to everything about the world from books and other text. Active exploration, mobility, and the ability to affect the environment are not necessary; the positions known as “embodiment”, “situationalism”, and “enactivism”(4) are not (IMO) valid. Their adherents have basically discovered that Reduction is necessary but the remedies they recommend, like embodiment, still don’t address how Reduction is done. You still need to learn from your activities and without some kind of Reduction, you cannot learn anything.

What matters here is how deeply this “world understanding” extends on top of the “language understanding”. Existing Deep Learning seems to have some problems getting deep enough. There are two positions one can take: “It will get better until it works” means we’ll get there incrementally. My own position is “Deep Learning isn’t quite sufficient and we need to find the next level technology beyond that” and so I’m working on exactly that. I have called this work “Artificial Intuition”, “West Pole Style Deep Learning”, and “Dynamic Deep Learning”. None of them are good; I need a better name, and I think I’ll switch to “Organic Learning”.

Why would that be a good name? Well, in traditional Deep Learning, a programmer designs the network topology using a programming language like TensorFlow, Theano, Torch, or Caffe. But in my kind of Deep Learning, the network starts out empty and grows “organically” to hold the understanding it is accumulating by reading the corpus. Basically, discovery of new knowledge means creating new “neurons” and “synapses” (well, their software counterparts in a simulation).

___________________________________

Chomsky has argued the existence of “universal grammar” by which he means “the system of principles and structures that are the prerequisites for acquisition of language, and to which every language necessarily conforms.” (5) He has argued that humans genetically inherit this knowledge. Do you agree with this view? How do you see AGI approaching the work of artificial language understanding?

Our cognitive capabilities outside of language already demonstrate a very powerful brain. But Chomsky’s principles and structures are not different from other principles and structures we need to understand the world, even without language. It is tempting to elevate humans to a position as the only language users but that would be incorrect. Alex the parrot knew over a hundred words and could count. His handler would show him a tray of objects and ask “how many red things” and Alex would answer “four”. Correctly. Koko the gorilla and other primates have demonstrated proficiency with sign language and specialized typewriters.

My take on Chomsky’s universal grammar is that our language capability comes about from having more of everything — neurons, synapses, training materials, feedback, and better control of our tongues — and that language understanding doesn’t need to be an inherited capability on top of general inherited ability to learn.

I am backing up this position by partial success in my Model Free systems that certainly do not contain a single line of code providing any information about the problem domain “for free from birth”. All language skills are learned from an unsupervised corpus. Full success would immediately overturn Chomsky’s argument by demonstrating that innate grammars are unnecessary.

___________________________________

You’ve stated that artificial understanding ultimately requires “soft science”, can you elaborate? How does code do the work of soft science? Is this not a dichotomy?

I have a short and a long answer. Computers doing AI are not solving computer-like problems. They are performing Reduction — Understanding work, normally done by the programmers. We must stop thinking of computers performing Reduction as “just computers” since normal principles of computing no longer apply.

AI systems are more like a programmer than a program.

The longer answer is that all programming is Reductionist. It is math, and Math is the manipulation of Models. But the world is Bizarre, which means a Model of the world as a whole cannot be created. It’s simply too large, to complex, and changes over time. The only valid viewpoint is “gather as much information as you can, continuously, and learn from it, and then jump to reasonable conclusions given your experience gathered over a lifetime, in every situation”

The distinction we need to make is that it’s OK to build a Model of a Mind, Reductionistically, using normal scientific programming techniques. Unit tests. The whole deal.

But when it is finished, it must not contain a single line of code that could be said to be part of “A Model of the World”. Or of language. Or whatever the problem domain is.

A Model of Mind can learn from the world and create its own Model of the World.

But any World Models we add ourselves as part of the programming of the system will limit our AI to operating in the part of the world we are modeling. The definition of “Narrow AI” now becomes “Contains human-crafted Models of (fragments of) the World”

AI was originally born out of Computer Science departments at our universities which in their turn came from Math departments. This was highly unfortunate, but totally understandable. The soft disciplines (psychology, evolutionary theory, etc) actually have much better theories of mind than the computer science crowd ever had. But I don’t think the soft sciences are the answer either.

Epistemology is “larger than science” and “stands outside of science”. It deals with exactly the issues that AGI research is trying to solve: What is knowledge, what is learning, how can anyone learn anything at all, how do we perform Reduction, what is Saliency and how can we program it in, how do we solve problems, what is Understanding, what is Reasoning, etc. Even “The Scientific Method” itself is defined in Epistemology, since you cannot “define Science” inside of Science. And you don’t have to read the old epistemologists. You just need to know the basics, and realize you are standing outside of Science when you are thinking about learning and knowledge, when using Neural Networks, and when building an AGI.

If this is a surprise to you, then you need to watch my videos. All of them.

What does it mean to “take hints from Epistemology” when programming an AGI? Here’s an example. The statement “You can only learn what you already almost know” is clearly a statement in Epistemology since it discusses knowledge and learning. We know we can’t teach calculus to first graders since the children haven’t learned enough basics of math.

But this is also the reason many Deep Learning starts by learning lower layers to a stable state before attempting to learn higher layers. If you allowed learning in all layers at once, then changes in lower layers would mean higher layers that relied of semantics of lower layers would be “fooled” by their lower layer sources, leading to problems with convergence and unpredictable results.

It is fascinating to note that most of the “Statements of Epistemology” that we need to use in AGI design elicit responses like “Well, that’s kind of obvious in retrospect and I never really thought about it that way”. We are normally uninterested in these things because we don’t know or even care how our brain does what it does. But when we are trying to duplicate what a brain does in a computer then we need to face all these questions head-on and to come up with answers we believe in and that can be implemented in the systems we are building.

So “scientific principles” such as “Do not use correlations; you must show causality” and “Do not jump to conclusions on scant evidence” may become major mental blockers to people entering the world of Holistic (Model Free) AGI research. Be scientific when programming the model-of-mind, but the program you are writing will have to do this jumping to conclusions about anything in the problem domain. It makes an ad-hoc Model of the World on its own and that’s the best it can do.

The extent to which this insight influences how we “Holists” write our code has some hilarious consequences such as “locking? We don’t need no stinking locking, the brain is flakey so we can be flakey too”. But that’s another show.

___________________________________

You’ve been in this field for many years. What is the one thing you would tell your younger self about AGI that you’ve learned in this journey?

Wait.

Our machines were too small until about 2005. Nothing we did before that really mattered. Reductionist AI (Models of small fragments of the world, AKA “Toy Problems”) were meaningless, and Holistic AI (Neural Networks) wasn’t possible.

We live in very exciting times. We have everything we need today to actually solve the problems that we need to solve. And we know which ones they are.

Starting with Reduction. Which we have already partially solved for several special cases, such as image understanding. We are now making progress every day.

To people starting out in AI today I’d like to say “Science is overrated; learn to think Holistically, and study Epistemology.” Science has been the main paradigm for extending our knowledge. But the emergence of Holistic tools like Neural Networks and general AI means that the idea that Science is the only game in town will come to an end in the next decade. It had a good run after 1650 and in 1850–2012. Yes, Science still makes sense in most domains. But there are certain problems in certain domains where the Scientific approach never made any sense, and we are starting to see why, and change is on the way.

The Reductionist Train is running out of track. The remaining hard problems, like those in physiology and medicine, in economics, in genomics, in language, and in AI all need an influx of more powerful tools. We need to extend the scientific toolkit with Holistic Methods before we can deal with systems with billions of moving parts, like people, cells, neurons, and words, where the connections matter to the point where the system becomes irreducible and out-of-bounds for Reductionism. I discuss this in my “Bizarre Systems” video.

___________________________________

I’ve written about the fallacy of “AI as an existential threat to humanity” and you seem to agree, in fact you’ve been quoted as saying the singularity is “impossible”.

Why do you believe this “super intelligence”(6) is impossible?

An “AI Singularity” is a silly idea, but slightly superior intelligences are possible. But there are limits to how good an intelligence can be. John McCarthy and Pat Hayes wrote the “Frame Problem Paper” decades ago where they pointed out that the world changes behind your back, so any world model you make is almost immediately obsolete. It will contain errors, and when the AGI makes decisions based on that erroneous information, it may well make a mistake. And if you make mistakes, then you are not infallible, at least. Yes, superhuman may well be in the cards, but the Reductionist pipe dream of an AGI that is a “Logic-based, fallacy-free, perfectly reasoning, omniscient, superhuman godlike intelligence” is simply impossible.

What matters more than the size and power of our computers and even the power of the ANN algorithms we use is the size, usefulness, and veracity of the training corpora. AGI evolution is limited by our ability to provide materials for them to learn from. Any ANN today learns a minuscule fraction of what any human knows, since we’ve been learning from birth. Humans “seem to” be able to do single-shot learning; in reality, we just already know almost everything the single shot learning experience is trying to teach us. Computers will get there, a little at a time, and will one day pass us by.

Recursive self-improvement is a myth.

It made sense as a theory in the Reductionist days when we envisioned all AIs as monstrously large systems with millions of propositions and thousands of errors… which the AGI itself would discover and fix. But modern ANN systems are defined with a few hundred lines of code. 200 lines of Tensorflow, done and done.

There’s nothing there to fix, recursively or otherwise.

We will therefore see a slow decades long succession of improved versions of lineages of AI projects, such as SIRI 4.0, 4.1, 5.0, etc. limited by human-controlled release cycles.

Decades of personal daily interactions with AGIs will allow us to comfortably get used to them.

And them to us.

___________________________________

endnotes:

(4) https://en.wikipedia.org/wiki/Enactivism

(5) In Wiktor Osiatynski (ed.), Contrasts: Soviet and American Thinkers Discuss the Future (MacMillan, 1984), pp. 95–101

(6) Bostrom https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom-ebook/dp/B00LOOCGB2