GPT-4 is here. An AI researcher’s take.
Prepare to be floored. GPT-4 is out now. As of March 14th 2023. We’re talking virtually alien technology.
I’ve been working on NLU (natural language understanding) off-and-on for 25 years. Trying to build parsers that break the famous Turing Test.
The Turing Test is smashed.
I don’t think anyone needs to bother actually running a test.
It’s self-evident that that measuring stick is irrelevant now. GPT-4 might fail the test because it’s too good at being human. It’s more like: is it super intelligence?
Practical updates? Think 50-page prompts. Incredible. That means you can get GPT to query or summarize a decent proportion of an entire book in one call.

Here’s the summary:
- Yes: 50-page prompts (input text) and 50-page completions (output)
- ‘Factuality’ is 40% up (less hallucination).
- Accuracy is way up. Way better than GPT-3 or 3.5. GPT-4 gets an A or B on almost any exam. This is documented. For a dozen real life exams.
- And GPT . . now accepts images as input, as well as text!
- Working with programming code jumps another level.
- Do not miss the tax accounting demo. Yes, tax. It demonstrates we’re in a new age. Virtually AGI.
- And BTW, Bing is already using GPT-4 as some guessed.
The release notes and white papers that come with GPT-4 are immense and exhilarating.
Here’s my very quick first take as a PhD deep learning engineer . .

Wow
It’s an impressive update and opens entire new possibilities, particularly with the large prompt size increase and — soon — accepting images.
Think, in addition to generating reports, essays and other content:
- summarizing entire chapters & books
- every programmer’s assistant
- cheap robot machine vision
- business intelligence via screengrabs
- your personal decoder of legal gobbledygook of all types
- the engine of early AGI
It’s kind of mesmerizing.
I think a common research application will be to pass GPT chapter-at-a-time of PDF books and ask it the questions that are needful to you.
It’s available in ChatGPT Plus or via API subscription in the playground (mode = chat, NOT mode = complete, model = gpt4) now, but the API itself is initially pretty pricey (see below).
This will come down, we can count on that, OpenAI have promised that. Remember how GPT-3 came down by a factor of 10 in price around 2 months ago? They want to lock in their market.
Probably the only thing missing — which would require some sort of a major architecture change — is enabling better semantic search with references/links reported. A bibliography. I can imagine that OpenAI might be leaving this to Microsoft perhaps to give their partner an exclusive advantage via an yet-to-come double upgraded Bing that fundamentally uses GPT-4 (??).
In any case, GPT-4 will be hugely beneficial for the world.
Model size?
BTW: What we don’t know yet is the size of the model . . . . ahhhhh, wait . . OK . . EDIT they’re keeping that a proprietary secret. From competitors. A little touch of business nous. And to make it harder for nefarious players to reproduce.
OK. First time they’ve done that. The last vestiges of the ‘Open’ of OpenAI have dropped. At least we get to pay to use the API.
50-page prompts!
That’s the biggest single feature for me.
Yes it means we can give large documents in one call and get GPT to do anything with it.
Summaries. Q&A. Essays. Analysis. Problem solving.
And the output is much larger too, it seems the same 50-page size as input ( = 25,000 words).
Image input

Here’s something we got a hint of a few days ago: GPT-4 will — incredibly — be multi-modal: accept input text and images although the image option will be a few weeks away but it was impressively demoed. There are no current plans to output images.
The example their webcast/blog gives is the ID of the items in the image by GPT-4 and answering ‘what is funny about the image’:
INPUT What is funny about this image? Describe it panel by panel.
OUTPUT The image shows a package for a “Lightning Cable” adapter with three panels.
Panel 1: A smartphone with a VGA connector (a large, blue, 15-pin connector typically used for computer monitors) plugged into its charging port.
Panel 2: The package for the “Lightning Cable” adapter with a picture of a VGA connector on it.
Panel 3: A close-up of the VGA connector with a small Lightning connector (used for charging iPhones and other Apple devices) at the end.
The humor in this image comes from the absurdity of plugging a large, outdated VGA connector into a small, modern smartphone charging port.
Impressive!
Endless possibilities.
Working with code
GPT-4 will re-write code in response to an error message generated by the compiler/interpreter.
Hello! I’m getting giddy. That demo on the webcast is incredible.

The accounting demo is the real winner
This will blow your socks off.
In the OpenAI webcast on YouTube, GPT-4 wades through a dozen pages of horrific tax law to calculate a couple’s tax liability taking 4 or 5 clauses – and the year – into consideration.
And explains why and shows all working, conducting flawless arithmetic (see below).
This is virtually non-conscious AGI. It’s the AI PA we’ve been waiting for.
This company should be worth more than $29B. Every human on earth would be nuts to not use it to speed up their work.

Separate ‘system’ steering & resource input
Now you can separate what is the instructions — known as ‘system’ steering — from an optional resource text.
In the past you just merged that together. Now it’s clearer to GPT what you are intending.

Scoring highly on everything human-like.
GPT-4 now hugely improves on ChatGPT and GPT3 on AI text metrics and doing human exams at mostly A or B levels.
On average GPT-4 is in the top 10% of human test takers. GPT-3 was in the bottom 10%.
That’s hugely better than GPT-3 and just incredibly impressive.

Less hallucination
GPT-4 demonstrates 40% higher factuality than GPT-3.5.

API Pricing
From the OpenAI email invite I got:
gpt-4 with an 8K context window (about 13 pages of text) will cost $0.03 per 1K prompt tokens, and $0.06 per 1K completion tokens.
gpt-4–32k with a 32K context window (about 52 pages of text) will cost $0.06 per 1K prompt tokens, and $0.12 per 1K completion tokens.
That’s fairly steep but expected to come down over time.
AGI
Non-conscious AGI is now less than a year away IMO, depending on how fussy you are.
Concluding remarks
The world is correctly going nuts over GPT, and soon specifically over GPT-4.
As an NLU researcher it’s definitely what we were aiming for the last 7 decades but never thought we’d actually see.
I’ve actually been predicting human-like NLU since the late 1990s to occur around 2020. I got that right. I always knew it didn’t need quantum computing. Just something ‘search-like’ that could interpret the knowledge embedded in the web text corpus. That was my thinking.
But I didn’t think we would go from almost nothing to arguably near super intelligent in 5 years!
GPT-3 way out performed GPT-2, for example in correctly explaining common sense reasoning tests.
Now GPT-4 has leap-frogged GPT-3, probably most impressively in the step-by-step carrying out of instructions.
Yeah, in an accounting, of all things, demo.
LOL.
Now to absorb this . .

Nice touch OpenAI . . LOL.

PS. My post is entirely me. Sorry. :)
Here’s today’s OpenAI webinar clip
I’ve started it at the awesome accounting demo. Feel free to rewind for all of it.
My recent posts
Fav posts of mine
Here’s my other articles on AI, ChatGPT & androids
ChatGPT enables smart apps but the bar is now sky-high for AI
ChatGPT is not wasteful of energy: 5 reasons
The crucial ChatGPT capability nobody is talking about
ChatGPT: Gary Marcus & Noam Chomsky are STILL in full delusion-mode
ChatGPT writes some pretty good stand-up comedy. LOL.
The entire field of AI is being professionally gaslighted by Noam Chomsky and Gary Marcus
The brain is WAY more like ChatGPT than symbolic AI: Gary Marcus? Noam Chomsky?
‘Consciousness’ means different things to different people . . and experts
ChatGPT: the stunningly simple key to the emergence of understanding
Enough already! Here’s a balanced view on human-like AGI via LLMs like GPT-3
ChatGPT: AGI by 2024, the hard part is now done
ChatGPT is awesome. But here’s a failure mode I discovered. LOL.
Revisiting emergence of intelligence in ChatGPT
AGI & generative AI like ChatGPT are (almost) everything now: here’s why
ChatGPT is an inflection moment in human history that debates can’t diminish
