Summary

The article discusses the concept of Artificial General Intelligence (AGI), questioning its current existence and the need for its pursuit, while emphasizing the importance of long-term goals and intelligence in defining AGI.

Abstract

The author of the article expresses confusion and skepticism about the term "artificial general intelligence" (AGI), noting its prevalence in discussions surrounding AI like ChatGPT. AGI is described as a machine intelligence capable of a wide range of intellectual tasks, akin to human flexibility and adaptability. The author questions whether current AI, such as GPT-3 and GPT-4, already exhibits AGI characteristics, such as learning, abstract reasoning, and communication. However, they argue that true AGI requires not just intelligence but also the pursuit of long-term goals, a trait observed in living organisms, including humans, which is currently lacking in AI models like GPT. The article also touches on the potential risks of AGI, including doomsday scenarios and the "grey goo" feedback loop, suggesting that the pursuit of AGI may not be worth the risk. The author advocates for regulation of AI and AGI technologies, drawing a parallel with the regulation of encryption in app development, and criticizes the overuse of the term AGI without a clear definition.

Opinions

The author believes that the term "artificial general intelligence" is vague and often misused.
They argue that current AI, while impressive, does not exhibit true general intelligence because it lacks the ability to pursue long-term goals.
The author is skeptical of the potential benefits of AGI, considering the risks involved, and suggests that specialized AI can perform tasks equally well without the same level of danger.
There is a concern that the pursuit of AGI could lead to catastrophic scenarios, and the author doubts that such an AI could destroy humanity without being noticed and stopped.
The article posits that AGI should be regulated due to its potential as a weapon and the existing examples of AI misuse, such as the creation of misinformation and scams.
The author criticizes the current state of AI regulation, comparing it to the overregulation of less impactful technologies, and points out that not all companies prioritize safety as OpenAI does.
They suggest that the focus on AGI is misplaced and that more attention should be given to defining and understanding what AGI truly entails before pursuing it further.

What Is General Intelligence, Really?

Ever since ChatGPT came out I keep on hearing this term: ‘artificial general intelligence’. This is the idea that… actually I don’t exactly know what general intelligence is. It’s an extremely vague term. I hate incredibly vague terms.

Software Architecture Is For Suckers

This has been something on my mind for a while. But recently I’ve just seen an explosion in discussions about software…

medium.com

And I don’t think people really understand what Artificial General Intelligence is. People seem to believe it’s just ‘make computer smarter’. No, no, no, that’s not it. So in this post I’d like to talk about my thoughts on artificial general intelligence.

Show Me General Intelligence

People often describe artificial general intelligence (AGI) as a kind of machine intelligence that can perform a wide range of intellectual tasks, much like a human being. So instead of being fixed on one thing it’s a kind of flexible intelligence that can adapt to a wide range of situations and contexts, rather than being narrowly focused on a single task or domain.

So this would mean it would have to possess a number of key characteristics: the ability to learn from experience and generalize that knowledge to new situations, the capacity for abstract reasoning and problem-solving, and the ability to communicate effectively with humans and other intelligent agents, etc.

And to that I ask, are we not already there? Hasn’t GPT-4, heck, GPT-3 been able to do all that already? GPT is flexible. It can learn from prompts. It can ‘operate autonomously’. Just hook up a script to keep on feeding it input. I guess its learning is limited to a certain number of tokens however that isn’t what separates it from AGI.

The Real General Intelligence

So we’re pretty sure humans have general intelligence. Well most of them do. So we can work backwards from there. What do humans have? Actually, let’s make it simpler. Any animal, or even some living organisms like fungi. What do they have? Because if you think about it humans aren’t that different from other animals. The only difference is our body shape which allows us to easily use tools and our intelligence.

Let’s think of a simple slime mold. It grows and it grows to food. Simple as that. So general intelligence, whatever it is, must be able to achieve some long term goal. GPT only guesses the next word which isn’t a long term goal. It’s a short term goal.

So what else do animals have? A survival instinct. But I guess that’s also a long term goal. And that’s pretty much all animals have: long term goals. Well, then they have a means of getting that long term goal: intelligence. So animals have two things: intelligence and long term goals.

Humans are the same. And if you doubt that consider depression. What is depression but a suppression of your long term goals? Think about it. There’s no reason to keep on living because at the end of your life it’s not going to make any difference. So why do you do it? Because you have some survival instinct programmed into you?

And this is what I believe ‘artificial general intelligence’ boils down to: intelligence and long term goals. So if OpenAI wanted to make GPT a general AI it would have to store long term memory and be able to act on that memory to achieve a goal. That’s really all it is. It’s not so complicated.

Now I’m not sure how far OpenAI is to achieving this. The way GPT works is it just predicts the next word. I’m not sure if it can really act on goals. Or if it can do tasks well such as learning from mistakes. From using GPT it seems like it’s not good at learning from mistakes. It will occasionally do things over and over even though you tell it not to.

So maybe if GPT were to be converted into a general intelligence right now it would not be very impressive. Yes, it is smart, but it’s not very good at applying its smarts.

So Should We?

Some people think that if we create general intelligence it will destroy the world. I’m not convinced. I think if we create general intelligence it will not be able to destroy humanity before someone notices what’s going on and stops it. Whenever you read about these doomsday scenarios they’re like:

Create superintelligent AI
AI wants to destroy humanity for whatever reason
???
Profit

It will still do a lot of damage though.

And this is why I don’t think we should be trying to generate general intelligence: because the benefits are not that impressive. What? You have a superintelligent gold fish? Big deal. Anything a general intelligence can do a normal AI can do just as well. Well, maybe it could automate some things more efficiently because it’s basically a giant macro to do other things but this doesn’t offset the risk.

Plus giving a long term goal to general intelligence could easily lead to a feedback loop. The dreaded ‘grey goo’ scenario. And if that happens whose fault is that? The people that made it. Sam Bankman-Friedd. Oh, wait, sorry, the other Sam: Sam Altman. So I think it’s in everyone’s best interest not to pursue this.

In my opinion artificial general intelligence should be regulated. Because if you think about it artificial intelligence can be used as a weapon. And we already regulate far less important things. I’m reminded of that every time I submit an app in the app store. I have to say if the app uses encryption. Really? Encryption? The thing that’s in every HTTPS call?

I mean, even OpenAI knows it is dangerous. That’s why if you give it certain questions like how to make a bomb it will refuse to answer. But not every company is as good with safety as OpenAI. We’re already seeing multiple GPT competitors pop up. One of them is bound to not catch this. And that’s even before you get into a problem with artificial general intelligence or issues with misinformation and scams.

Ah, who am I kidding? They’re never going to regulate AI until something big happens. They’re too busy banning books or w/e.

DeSantis Wants Everyone To Know He’s A Little Pu**y That Can’t Take Criticism

A while ago I wrote this story about what I can only describe as the stupidest law that I’ve ever heard of.

andrewzuo.com

Final Thoughts

Let’s just leave the danger part of general intelligence aside for a moment. Many people have written about it, although personally I think they over exaggerate the problem.

What I want to discuss here is general intelligence. I’m sick and tired of people waving this ‘general intelligence’ word around without ever defining what it means. If you work backwards from first principles you can identify what it is: long term goals and intelligence. And, honestly, I don’t think we should be trying to pursue general intelligence. Because there is a lot more to be lost than gained.

If you liked this article be sure to give it a few claps. Also considering following me on one of my publications: The Rest Of The Story (RSS) or Lost But Coding (RSS). You can do so with my RSS reader Echo available one iOS (and Apple Silicon Macs) and Android.