AI ARTICLE
ChatGPT Can Now See, Hear, And Speak
ChatGPT just got some major upgrades from OpenAI — now the AI can see and hear you!
On Monday, OpenAI shared that ChatGPT will soon analyze images and have real conversations using voice on the mobile app.
This is thanks to new capabilities with its GPT-3.5 and GPT-4 models.
Over the next two weeks, these features will roll out to Plus and Enterprise users. Voice will work on iOS and Android, while image recognition is coming to mobile and web.
The image feature lets you show ChatGPT one or more photos to chat about. OpenAI says this can help with daily stuff like figuring out dinner from pantry pics or troubleshooting issues with your grill. You can also circle part of an image on your touchscreen so ChatGPT focuses there.
In a promo video, they show someone asking how to raise their bike seat while showing photos, manuals, and toolboxes. ChatGPT then walks them through it. I haven’t tested it myself though, so we’ll see how well it really works.
OpenAI hasn’t shared the nitty gritty details, but basically these multimodal models can process text, images, and other data all through the same neural network. The AI likely uses something like CLIP to bridge text and images so ChatGPT can understand them together. But that’s just my guess based on similar research.
For voice, ChatGPT now allows back-and-forth conversation powered by a new text-to-speech model. Users can turn it on in settings, then pick one of five voices with names like Juniper, Sky, Ember, etc. OpenAI partnered with voice actors to create them.
The Whisper speech recognition they already use will transcribe your words into text for ChatGPT to process. Whisper’s been built into the mobile apps since they launched.
Alright, so that’s the scoop! ChatGPT’s getting some cool upgrades to see, hear, and speak. We’ll have to try it out once they roll out to see how smart it really gets. As always, thank you for reading!
Source: https://openai.com/blog/chatgpt-can-now-see-hear-and-speak
If you like the article and would like to support me, make sure to:View More Content📰| Subscribe 📬| Follow Me On LinkedIn, Twitter💙
Check out My curated articles lists :