AI
The Surprising Things ChatGPT Can’t Do (Yet)
At this point, most of us have seen amazing examples of ChatGPT and its abilities. Everyone is eager to find out what it can do. But to understand where you can and should use it, it is also necessary to look at its current limitations and where it is likely to fail. In this article, I’ve tasked ChatGPT with different questions to see what it cannot do. Let’s get into it:
The Puzzle
I previously had a few puzzles made for GPT-3 (prior to ChatGPT) using the command line:
To my disappointment, it failed one of the puzzles most of the time, and another it could not solve at all. Now, it is ChatGPT’s turn. I decided to test it on the puzzle that GPT-3 completely failed to solve. I also removed the part involving the command line and simply showed the parts of the puzzle immediately:
Here, mapping the number into letters should form the words data science. What does ChatGPT answer?
Not even close. Let’s see what happens if I add clear instructions for what needs to be done:
Now, it got it right. It should be easy for a human to solve this puzzle without any instructions since there aren’t that many options to choose from. ChatGPT seemingly can’t deduce this.
The Reading Test
In this experiment, I decided to test if ChatGPT could make logical conclusions from a conversation:
Since the initial statement says that one of the people he is talking to is his father, the only people it could be is Douglas or Josh. The first rather obvious clue is that Josh says “Good job son”. In addition:
- Josh says “tell your mother” and “we should eat them”
- Douglas replies “I wish my family would eat fish tonight, my father is making pancakes”, implying that he is not part of the family that will eat fish.
Yet, asking ChatGPT several times, it always answered:
Strangely, it persisted in guessing that Douglas was the father. I wonder why.
The Trick Question
Next, I formulated a trick question:
The trick is that after the first day the 10 coins will have disappeared and turned into 20 apples. Thereafter, there are no coins that can be transformed. Thus, the answer will still be 20 after 3 days. Can ChatGPT figure this out?
Out of 4 answers, all were wrong. Three of them were different (shown above). Arguably, the question could be seen as ambiguous. Thus, I tried to alter the question to make it more clear with formulations such as the following:
Asking it numerous times, none of the formulations caused it to always answer correctly. It answered correctly only about 20% of the time, thus, the majority of answers were incorrect.
Summary
While there has been a lot of talk about the prowess of ChatGPT, it’s also useful to understand its limitations. Clearly, from the examples, it’s still far from a human level.
What is interesting though, is that it sometimes can respond with incredibly sophisticated answers to difficult questions, yet other times fail at simple tasks. My guess is that ChatGPT is mostly interpolating, i.e. it can solve things well that it has seen or is close to what it has seen.
But asking novel questions, requiring extrapolation, i.e. questions outside of its training data, seems to cause it to fail more often. There is still a long way to go towards artificial general intelligence.
