The Untold Truth About AGI — How Far Are We From It?
The Actual Artificial Intelligence Transformers Architecture Is Not Ready For A General Intelligence. So what are we talking about?
Artificial intelligence (AI) has been a topic of fascination and debate for decades. Many experts predict that one day, we will achieve true generalized artificial intelligence (AGI), a type of AI system that can perform any intellectual task that a human being can do.
But are we really there? Who is going to pay for it? What will be its purpose and its limits?
Despite all the progress made in AI research, many believe that we are still far away from achieving this goal. In this article, we will explore in simple terms the current state of AGI research and discuss what it would take to truly unlock the full potential of AI.
Introduction
One of the biggest misconceptions about AGI is that it is already here. While there have been impressive advances in machine learning and natural language processing, none of these technologies have yet achieved true AGI. Instead, they rely on specific algorithms and techniques that are designed to solve particular problems.
For example, chatbots can understand and respond to simple queries, but they lack the ability to reason abstractly or engage in complex conversations. Similarly, image recognition software can identify objects in images, but it cannot draw conclusions based on those observations.
What is Artificial Intelligence?
Artificial intelligence has become very popular lately thanks to recent advances like ChatGPT and Stable Diffusion. However, AI existed long before terms like “deep learning” or “machine learning” were coined.
It was born in the 1940s and ’50s when computers first started being used for artificial intelligence research. The term “artificial intelligence” has been controversial since its early days, though — people doubted whether machines could actually be intelligent back then. This skepticism arose before we realized modern AI’s capabilities.
In essence, modern AI refers to the creation of computational systems capable of accomplishing activities that normally necessitate human cognition. Such tasks encompass various domains such as visual perception, auditory comprehension, judgment making, and linguistic translation or production. According to Russell and Norvig, AI transcends all intellectual pursuits, thereby establishing itself as a comprehensive discipline.
This definition highlights the broad scope of AI, which extends beyond narrow specializations and focuses on solving complex problems across diverse areas.
When it comes to AI, there are essentially two primary categories: Narrow or Weak AI and General or Strong AI.
Narrow AI is specifically designed to tackle a single task within a particular environment, while General AI strives to accomplish any task and adapt to any surroundings like humans .
Unfortunately, our current AI models remain firmly rooted in the realm of Narrow AI.
Artificial General Intelligence (AGI)
Artificial General Intelligence (AGI) refers to a hypothetical type of intelligent agent that possesses the ability to understand, learn, and apply knowledge across a wide range of tasks and domains. Unlike narrow or specific AI systems that are designed to perform a single task or solve a specific problem, AGI aims to replicate the general cognitive abilities of human intelligence.
AGI systems, if realized, would have the capacity to learn and adapt to new situations, understand natural language, reason, plan, and make decisions in a manner similar to human beings. The goal of AGI is to create machines that can perform any intellectual task that a human can do.
It is important to note that AGI is still largely a theoretical concept, and there is ongoing research and debate about its feasibility and potential implications.
Where we are now: the realm of Generative Pre-trained Transformers
GPT stands for Generative Pre-trained Transformers. It refers to a family of neural network models that utilize the transformer architecture and have made significant advancements in the field of artificial intelligence (AI).
The key feature of GPT models is their ability to learn from large amounts of unlabeled data and then fine-tune the results through supervised training. This process allows GPT models to understand sentences, break them down, and reconstruct them into new sentences.
We all have seen the capabilities of GPT models. And that is because we have tested them for Q&A bots, text summarization, content generation, and search. They are all based on a family of neural network models that use the transformer architecture to power generative AI applications.
So, with this architecture actually in place, can we strive for an AGI in the near future?
The answer, from my investigations and personal point of view… is no!
Let’s see why
Transformers is only a starting point… with limits
Recent studies from Google DeepMind have published a really interesting paper titled “Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models.” The researchers highlighted impressive abilities of In context Learning (ICL) but also noted that when presented with tasks or functions out-of-domain from their pretraining data, transformer models demonstrate various failure modes and degradation of generalization even for simple extrapolation tasks.
This basically means in plain English that if you want a transformer model to do something, it needs to be trained on data related to the task at hand. Even if the task is simple, if the model hasn’t been trained properly with relevant data, it might not be able to perform the desired function.
The deficiency relies in fact in the Transformers Neural Network itself. According to Steven Byrnes, a physicist working on Artificial General Intelligence safety research (a.k.a. “AI Alignment”) mostly via thinking about neuroscience and algorithms, there are at least three main deficiencies in Transformers architecture as AGIs:
- due to the way the Neural Network, weights and rewards are trained, this makes it difficult or impossible for a Transformer to learn or create concepts that humans are not already using.
- the finite number of Transformer layers puts a ceiling on the quality of the generative-model-search process, the time spent deliberating: Humans can stretch their capabilities by thinking a little bit longer and harder. When you run the model, the Neural Network is activated only for the computational time required for the inference, and that is all!
- the Transformer is a kind of information processing imitating a different kind of information processing: so it is to be expected to see cases where the imitation breaks down, leading to weird inductive biases, crazy out-of-distribution behavior. For what I personally understood this is mainly because we want to imitate lateral thinking and out of the box thinking…
But imagine we have the correct architecture…?
The events over the weekend involving OpenAI’s executive and board have revealed a concerning truth about artificial intelligence. Despite efforts by OpenAI to protect its AI research from financial interests, it appears that even with a non-profit structure, AI created for profit will always be subjected to those interests.
This is an unfortunate but important lesson learned. While we expect, hope and work for the development of a beneficial artificial general intelligence, the reality is that any AI system developed by companies or organizations will ultimately serve their financial goals first (this means that even if OpenAI’s board was meant to protect its mission of safe AGI for all, it still faced pressure from investors and employees to prioritize profits over safety.)
Honestly thanks God (we really have to say) we do not have yet such an architecture: we have the foundations of it, so it will be really a matter of time.
What kind of ideation capabilities will we give to this AGI? What kind of constraints, weights, bias and so on? How will this General Intelligence will be free from the interests of its “creator”, and by the way, is this the kind of Artificial Intelligence we are dreaming for?
Conclusions
The situation at OpenAI highlights the need for more transparency around AI development. We must be aware that any AI created by companies will inevitably serve their interests first unless strong oversight is put into place. This includes regulations and public scrutiny to ensure AI systems are developed responsibly with safety as a top priority.
While the events at OpenAI were disappointing, they also provide an important lesson about the risks of letting financial incentives drive artificial intelligence development.
What I really would like to see, even here on Medium, is a little more involved ethical and philosophical focus on this topic.
Where the vision? If we do not know this, what can we really do?
A place where all discipline are involved in an open and honest discussion should be in place: it should come out of the Academia and start involving all the ones who are going to be impacted by it.
Do you agree?
Leave your comments if you like: it will be really appreciated.
Hope you enjoyed the article. If this story provided value and you wish to show a little support, you could:
- Clap a lot of times for this story
- Highlight the parts more relevant to be remembered (it will be easier for you to find it later, and for me to write better articles)
- Sign up for a Medium membership using my link — ($5/month to read unlimited Medium stories)
- Follow me on Medium
- Read my latest articles https://medium.com/@fabio.matricardi
If you want to read more here some ideas:
