Size Matters And Facebook Just Unleashed A Beast
Meta/Facebook’s Open-Source AI Model Outperforms OpenAI’s GPT-3
Meta/Facebook released a collection of new AI models this week.
One of them, LLaMA-13B, surprised the AI community when it found that it outperformed OpenAI’s highly acclaimed GPT-3 in most benchmarks (the tech-savvy can read the original paper here).
But that’s not all! Meta’s LLaMA does not only outperform OpenAI’s flagship model, but it’s also more than 10× smaller!
Oh, and it’s basically open-source. That, too, is an absolute first.
Does Size Really Matter?
Yes, in AI size matters a lot. That’s why everybody keeps talking about parameters. Now, a parameter is basically like a variable that a machine-learning model can use to learn and make predictions. The more parameters the model has, the better it can do its job, but it also needs more space and energy to use them.
So, if a model can do a great job with fewer parameters, that’s obviously a great step forward because it saves a lot of time and energy:
- GPT-3, the foundational model behind OpenAI’s flagship ChatGPT, has 175 billion parameters.
- Meta released a whole collection of models last week, ranging from 7 to 65 billion parameters, with the most astonishing one being LLaMA-13B, the 13 billion parameters model outperforming the 10x bigger GPT-3.
As mentioned earlier, this means that a model 10 times smaller does almost the same work! Which could now lead to the development of powerful AI models running locally on our laptops or smartphones.
An Unexpected Move
Understandably, this was one proud tweet when the social media and Big Data pioneer announced that its newly released AI text generation model would outperform its major competitors:
Furthermore, and unlike other successful models such as Chinchilla, PaLM, or GPT-3, Meta used only publicly available datasets to train their model. This means that their work is actually reproducible and ironically more open-source than anything OpenAI has released — a rather unexpected move from Meta, indeed.
It will be exciting to watch how this plays out.
No doubt, we’re now definitely in for a treat with this crazy fast AI race!
For more information about AI & Creativity, follow me on Twitter or Medium (use my referral link to get full access to all my articles and those of thousands of other writers).
If you like my content, why not leave a “clap” at the end of this article, so more people can see it?
