StableLM And The Open Model Movement
Why It Matters For The Future Of AI
The open source vs proprietary software battle has been going on since the age of time.
And in case we need a refresher… Linux won, and WordPress still powers most of the websites on the internet.
but seriously, without open source, many of the technologies we take for granted today would never have been developed.
When DALL-E 2 was released last year, that was amazing, but it was Stable Diffusion that ignited the revolution of image transformers. I would guess there are probably thousands of image tools and apps that run on top of Stable Diffusion today.
It is arguable that having both open source and proprietary software in our world is necessary, but we mustn’t underestimate the power of open source to democratize innovation.
Additionally, why would we want the most powerful technology in the world to be fully controlled and managed by a few big tech companies?
Haven’t we learned our lesson yet?
The good news is open source LLMs are having a moment. Here’s what’s been happening…
Meta’s LLaMa
What if I told you that Meta, the company nobody trusts with their data, is one of the most open AI companies in the world, and OpenAI is actually one of the most closed?
In February, Meta released LLaMa, a large language model primarily for researchers who have limited resource capabilities. And while it is open source, Meta released LLaMA under a non-commercial license, so it can’t be used for business-related purposes.
Soon after, Stanford University’s Center for Research created the Alpaca model, an instruction-following language model that is built on Meta’s LLaMA. The Alpaca model is designed to increase the accessibility and affordability of large language models.
The Open Source ChatGPT
With the popularity of ChatGPT and conversational bots, we now have an open source alternative called Open Assistant.
An assistant that not only understands tasks but can also interact with third-party systems and dynamically retrieve information to complete those tasks.
The project is built using Python, TypeScript, and JavaScript, and it’s all part of the LAION (Large-Scale Artificial Intelligence Open Network) initiative.
The best part is that the code and models are licensed under the Apache 2.0 license, and the training data will also be released under CC BY 4.0, which means that it’s free to use and modify.
Enter StableLM
But just like Stable Diffusion revolutionized the image model movement, we now have an open-source model by Stability AI that aims to do the same for natural language models.
According to Stability AI, their StableLM model has been trained on a bigger version of the Pile, which is an open-source dataset that combines information from various sources such as Wikipedia, Stack Exchange, and PubMed.
At the moment, StableLM models with 3–7 billion parameters are already available, while larger ones with 15–65 billion parameters are expected to arrive later.
You can try a demo of it in Hugging Face.
Just like ‘Open Assistant’ is an open source version of ChatGPT, Stability AI is also releasing an open-source competitor called ‘StableChat’ — soon to be released.
The Importance of Open Source AI
If you are a company or startup that is building your business on a proprietary foundation model, you run the risk of following the rules associated with the model owner.
You are paying rent to live there.
I know OpenAI is a respectable company and GPT-4 is a great LLM, It’s probably totally fine to build your business on OpenAI’s APIs and plugins. But if that’s all there is, we would have no choice but to comply with OpenAI’s rules, prices, and conditions.
Why would we bet the future of the internet on proprietary systems?
Final Thoughts
Open source provides innovation from anyone and anywhere, it is an equalizer. A grassroots bottom-up movement as opposed to a corporate strategy imposed on society.
AI by the people for the people.
If you liked this article, throw out some Medium love… claps, comment, and be sure to follow.
You can also support my work on Medium by becoming a member using this referral link.
