What OpenAI Doesn’t Want You To Know: The Deleted Sam Altman Article
Insights from a Discussion with OpenAI’s CEO
On May 29th Raza Habib posted a short article with key takeaways from a meeting with “Sam Altman and 20 other developers to discuss OpenAI’s […] product plan”.
48 hours later the article was deleted with the comment: “Too much alpha in there. Taken down.
So, what was the “alpha” that was deemed too powerful to be left in the public domain?
Well, the insights from the deleted article are not only informative but also serve as a guide to understanding the course AI might take in the coming years.
We will be talking about:
- 1M token context windows
- open-sourcing GPT-3
- a delay for ChatGPT plugins
- community-supplied models
- $100.000 private copy of OpenAI models, and more …
👀 So, let’s take a closer look… thanks to the Wayback Machine Internet Archive, we still have access to the original post from May 29 (link at the end of this post).
GPU limitations
“OpenAI is heavily GPU limited at present”
The most pressing issue, as per Altman, is OpenAI’s current GPU limitations. Due to GPU shortages, their progress has been slowed significantly.
In this context, the following was mentioned:
- There is the possibility of expanding to 100k — 1M token context windows within a year (!), but significant research breakthroughs is required for larger context sizes.
- The API for fine-tuning is hindered by the lack of GPU availability. However, support for fine-tuning is expected to improve in the future, possibly including a marketplace for community-supplied models.
- OpenAI offers a service where customers can obtain a private copy of the model (currently limited by GPU availability) which comes with an upfront payment of $100,000.
OpenAI’s Roadmap
With the above-mentioned limitations, Altman laid out OpenAI’s provisional near-term roadmap:
- cheaper and faster GPT-4,
- longer context windows,
- extended finetuning API
- a stateful API
- beyond 2024: multimodality will be key focus (if there are enough GPUs)
Regulation and Open Source
In the debate between regulation and open source, Altman shared his belief that while regulation for future models is needed, existing models are not dangerous and should not be regulated or banned.
Interestingly, he also mentioned the consideration of open-sourcing GPT-3, highlighting the organization’s commitment to the open-source community.
ChatGPT Plugins
In regards to ChatGPT plugins, the discussion revealed that they might not be introduced to the API soon. Despite the demand from developers, Altman expressed skepticism about their product-market fit.
On the other hand, Altman reassured that other than ChatGPT, OpenAI won’t release more products, alleviating concerns about OpenAI becoming a competitor.
The Scaling Hypothesis & AGI
OpenAI will continue to focus on making its models bigger because data shows that larger models continue to improve in performance.
At the same time, Altman acknowledges that scaling cannot continue at the current pace, but will be slower.
However, even if the scaling rate slows down, that means shorter timelines for AGI (Artificial General Intelligence) development.
Link to the full article on Wayback Machine: https://web.archive.org/web/20230531203946/https://humanloop.com/blog/openai-plans
🚨 For more information about AI & Creativity, follow me on Twitter or use my referral link to get full access to all my Medium articles (+ those of thousands of other writers).
➡️ If you like my content, why not leave a “clap” at the end of this article, so more people can see it?