MPT-30B’s release: first open source commercial API competing with OpenAI

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

1445

Abstract

preview, and Bard, which is still in beta.</li><li><b>Data & Model control: </b>The release of MPT30 as both an infrastructure and an API is enticing for businesses seeking full control over their data and models; it offers a streamlined process for training and utilizing their models online.</li><li><b>Model Settings:</b> With 30 billion parameters, this model is capable of running on a single A100 GPU. It offers an 8k-token context window, which is the largest compared to open-source models, and even surpasses the 4k-token limit of GPT-3.5-turbo-0613. For comparison, Falcon40B is currently limited to a 2k-token context window.</li><li><b>Performance</b>: the blog post release states that it is comparable to GPT-3 davinci-003 (which is very decent and can be used for various tasks such as categorization, data extraction, sentiment analysis and so on). It also states that it is competitive with other open-source models such as LLaMa-30B and Falcon-40B. However: it doesn’t say much on how well it performs on different languages. <i>My assumption is that its performance would drop very quickly on languages other than English</i>.</li></ul><figure id="97b6"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*Et1Stceg7xMJaG1J.png"><figcaption>Performance comparaison with Falcon 40B, current Open Source LLM leader</figcaption></figure><h1 id="9f96">What does it mean for the market?</h1><ul><li><b>Pricing</b>:

Options

it’s hard to have a fair price comparaison since Mosaic Inference packages are based on a cost per “GPU hour” and OpenAI is based on the number of tokens used. Yet the blog post states that a same operation (e.g., serving a fined-tuned model via API) would be <b>x15 cheaper through their infra compared to OpenAI’s GPT-3 APIs.</b></li><li><b>Worth it? </b>if you are relying on a davinci fine-tuned models, obviously this would offer some significant savings! Nevertheless, GPT3.5 turbo is 20 times cheaper than davinci models and performs much better without fine-tuning ; it can be worth comparing both approaches first. Note that GPT3.5 turbo has a 16k version as well, which can further extend its capabilities.</li></ul><figure id="8c04"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*AWU3XAhk-Yzjo3r3.png"><figcaption><i>“Hosting a finetuned model using our Enterprise tier can offer up to 15x savings vs comparable OpenAI APIs. Finetuned models also offer better output quality than off-the-shelf models.”</i></figcaption></figure><h1 id="9cd9">Conclusion</h1><p id="3a03">This is a significant development primarily because the open source community is rapidly gaining momentum in competing head-to-head with OpenAI. MPT30B is merely the initial step. We can expect to see more advanced, efficient, and cost-effective models emerging soon, positioning them as viable alternatives to OpenAI models.</p></article></body>

MPT-30B’s release: first open source commercial API competing with OpenAI

Is it worth it? Can it replace OpenAI’s APIs? Everything you need to know! Update: mosaicML , the company founded in 2021 behind MPT-30B was acquired by Databricks for $1.3B!

MPT-30B was just released by MosaicML for commercial use! and it’s a big deal! How does it compare to other models and is it cheaper than OpenAI?

But first, if you like this topic, please consider supporting us: 🔔 clap & follow 🔔.

I also share a weekly newsletter with an in-depth analysis of recent development, and you can find me on twitter.

Why is it a big deal?

Proprietary vs. Open Source: OpenAI is dominating the market with very few competitors, such as Google’s PaLM, which is in public preview, and Bard, which is still in beta.

Data & Model control: The release of MPT30 as both an infrastructure and an API is enticing for businesses seeking full control over their data and models; it offers a streamlined process for training and utilizing their models online.

Model Settings: With 30 billion parameters, this model is capable of running on a single A100 GPU. It offers an 8k-token context window, which is the largest compared to open-source models, and even surpasses the 4k-token limit of GPT-3.5-turbo-0613. For comparison, Falcon40B is currently limited to a 2k-token context window.

Performance: the blog post release states that it is comparable to GPT-3 davinci-003 (which is very decent and can be used for various tasks such as categorization, data extraction, sentiment analysis and so on). It also states that it is competitive with other open-source models such as LLaMa-30B and Falcon-40B. However: it doesn’t say much on how well it performs on different languages. My assumption is that its performance would drop very quickly on languages other than English.

Performance comparaison with Falcon 40B, current Open Source LLM leader

What does it mean for the market?

Pricing: it’s hard to have a fair price comparaison since Mosaic Inference packages are based on a cost per “GPU hour” and OpenAI is based on the number of tokens used. Yet the blog post states that a same operation (e.g., serving a fined-tuned model via API) would be x15 cheaper through their infra compared to OpenAI’s GPT-3 APIs.

Worth it? if you are relying on a davinci fine-tuned models, obviously this would offer some significant savings! Nevertheless, GPT3.5 turbo is 20 times cheaper than davinci models and performs much better without fine-tuning ; it can be worth comparing both approaches first. Note that GPT3.5 turbo has a 16k version as well, which can further extend its capabilities.

“Hosting a finetuned model using our Enterprise tier can offer up to 15x savings vs comparable OpenAI APIs. Finetuned models also offer better output quality than off-the-shelf models.”

Conclusion

This is a significant development primarily because the open source community is rapidly gaining momentum in competing head-to-head with OpenAI. MPT30B is merely the initial step. We can expect to see more advanced, efficient, and cost-effective models emerging soon, positioning them as viable alternatives to OpenAI models.