Google Bard First Impressions — Will It Kill ChatGPT?
I was among the first people to get hands-on access to Bard. Here are my thoughts.
Yesterday evening, I received an exciting email from Google — I was invited to be one of the first people to test Google Bard, Google’s soon-to-be-released AI chatbot.
Bard is Google’s answer to the wildly popular ChatGPT system from OpenAI, which has quickly become the fastest-growing app in history. Since ChatGPT launched late last year, OpenAI has continuously improved the system — their GPT-4 update last week was a major leap ahead in terms of the system’s capabilities.
Google, of course, does not want to be left in the lurch. That’s especially true since their major competitor in the search space, Microsoft’s Bing, already includes AI features driven by GPT-4. Loath to let another giant company eat their highly-lucrative search engine pie, Google is now rolling out Bard in an effort to compete.
So, how does Bard look? What’s it good at, and where does it struggle? And will it kill ChatGPT? Let’s explore.
Here’s a video that I made sharing some of my first impressions in the hours since I received my Bard invitation.
The Interface
Let’s begin with Bard’s interface. Upon receiving my invitation to test Bard, I clicked through to the app, accepted some Terms of Service, and got my first glimpse of Bard’s interface.
Like many Google products, Bard’s interface is clean and clutter-free. There’s a text window, a place to enter prompts, and not much else.
Notably, there doesn’t appear to be a chat history tab, as you get with ChatGPT. That means you can’t easily go back and retrieve your old Bard conversations. These conversations are apparently more ephemeral.
Overall, the Bard interface is simple and user-friendly. Whereas the ChatGPT interface is more functional and business-like, Bard feels more like a tool geared towards people who land at Google expecting to execute a search.
That’s probably indicative of the fact that OpenAI didn’t expect ChatGPT to be such a sensation. They thought it was a small, incremental improvement over their existing models and likely designed the interface more to appeal to Beta testers than actual users. If you somehow built a car that ran on nuclear fusion and produced no emissions, you wouldn’t worry about its paint color.
With the flurry of excitement around ChatGPT, they’re not about to spend precious time updating the UI. Conversely, Google seems to have built the UI a bit more slowly and deliberately.
Speed
Bard is fast. To test its speed, I asked the system to “write 500 words about Bichon Frises.” It responded in 9.5 seconds with a big chunk of text.
The text is actually 329 words (Large Language Models aren’t great at counting), but under 10 seconds is a lightning-fast response time. Bard also delivers its responses all at once, whereas ChatGPT types them out as if it’s a person writing into a word processor.
I gave the same prompt to ChatGPT Plus, running GPT-4. It took 3 minutes and 2 seconds to respond with 428 words.
In short, Bard is way faster. Here’s their respective Words per Minute rates, based on my testing:
- ChatGPT: 141 words per minute
- Bard: 2,192 words per minute
That’s a massive difference. Still, it’s unclear whether that advantage will last. Bard is only available to a handful of Beta testers, whereas ChatGPT is continuously used by up to 100 million people worldwide. That’s a wildly different server load to deal with.
Once Bard gets more users, we’ll have to see if it retains its speed advantage. For now, though, Bard is much faster than ChatGPT.
Did you find this analysis helpful? Sign up for my Medium newsletter to get my stories in your inbox daily for free
Capabilities
So far, it sounds like Bard is winning — it’s both prettier and faster. But when you get into the actual content it generates, the system’s performance degrades quickly.
Bard’s interface is littered with disclaimers about its capabilities.
Google is very clear that “some of the responses may be inaccurate” and also that Bard has substantial guardrails.
These include:
- The inability to write code. Google says “Bard can’t help you with coding just yet. Bard is still learning to code, and responses about code aren’t officially supported for now.”
- “Built-in safety controls” that prevent malicious or dangerous answers
- Limitations that appear to cause Bard to refuse to write a response, rather than writing a response that might be incorrect or beyond its abilities.
Whereas ChatGPT will generally attempt to do nearly anything you ask of it — and often will succeed — Bard is much more comfortable saying “No” or simply telling you that something is beyond its capabilities. You can see some examples of that with my real-world tests below.
I tried several “jailbreaking” prompts with Bard, and it refused to take the bait.
But the downside is that it also refuses to do a variety of tasks that are child’s play for ChatGPT. For example, I asked it to respond to a sales email I received, using my voice. Bard simply refused.
ChatGPT obliged with a customized message that talked about my needs as a writer:
I can understand refusing to give instructions for building a bomb or something equally nefarious. But refusing to write a response to an email? Either Bard is seriously lacking capabilities that other LLMs have, or Google has substantially clipped its wings in its early testing phase.
Real World Test — Writing a Blog Post
Asking Bard to write a blog post yielded similar results. Specifically, I asked Bard to write a post answering the question “Are Bichon Frises Hypoallergenic?”
You can see the results in the video above.
It responded almost instantly with the trappings of a post. But the information was bland and generic. Nominally, its response consisted of words, but there wasn’t much in the way of organization, flow, or truly helpful information.
The response felt more like a summary of data pulled from the top-ranking pages on Google, rather than something original and helpful. It also started rambling with generalities about Bichons.
ChatGPT, on the other hand, responded with a post that had a clear introduction, narrative structure, and conclusion. It was truly useful, and something I would actually read.
ChatGPT’s post also fit the intent of my query much better. And Bard stumbled further when I asked it to make some updates to the post, by including links to relevant articles and turning headings into H2 tags. The system simply refused, whereas ChatGPT performed these tasks cheerfully and accurately.
Real World Test — Answering a Search Style Query
For my next test, I decided to take a stab at asking Bard a search-style query. Google dominates the search space, so I hoped the model would do better than its competitor here.
To test its search abilities, I gave Bard the prompt: “What are some fun things to do in Walnut Creek, California?”
Bard responded with a bulleted list. Again, the information was nominally accurate (it included generalities about Walnut Creek activities), but on closer inspection, lots of things were off. For example, Bard advised me to take a “bike ride through the Lindsay Wildlife Experience.”
Lindsay Wildlife is an indoor museum. I doubt they would appreciate me riding my bike through there!
In contrast to ChatGPT’s responses, Bard’s response were also much more generic. Whereas Bard told me to:
“Visit the Ruth Bancroft Garden, a beautiful botanical garden with a variety of plants from around the world”
ChatGPT wrote:
“The Ruth Bancroft Garden: A beautiful and unique 3-acre garden showcasing a diverse collection of succulents, cacti, and drought-tolerant plants from around the world, it’s a must see for plant enthusiasts and photographers.”
The second response is much more specific and interesting. It captures the uniqueness of this attraction (the garden is devoted to drought-resistant native plants and is a popular photo spot) instead of simply calling the garden “beautiful” and saying it has “a variety of plants.”
Surprisingly, ChatGPT did a better job at responding, even with explicitly search-oriented queries.
Conclusions
At least from my initial testing, I walked away feeling that Bard shows potential, but that it’s currently far behind ChatGPT in terms of overall capabilities.
Bard today feels like it’s where ChatGPT was as of GPT-3. It generates text that’s somewhat accurate and specific to the query, but it falls flat with its understanding of intent and human knowledge.
Bard also seems far more conservative and safety-conscious than ChatGPT, even in its first iteration. ChatGPT was always happy to try giving a response, whereas Bard is much more comfortable saying “no.”
That’s likely reflective of Google’s principles of AI innovation, which emphasize safety and scientific excellence above all else. Perhaps applying that high level of rigor is good for society, but it makes the tool itself much less useful for people who are trying to improve their productivity.
Overall, from my testing it appears clear that Bard isn’t attempting to compete with ChatGPT as a tool for general applications. ChatGPT can do nearly anything, from composing poetry to writing software, to reformatting data.
Bard feels like it’s much more tailored to a specific task: search. Even though it falls flat on that task now, with more training data and more time, its responses will almost certainly improve.
And once it does, Bard will have several advantages in the search space:
- Its fast response times, assuming they remain even once the system is used at scale, are more compatible with a search or voice query response. Waiting 10 seconds for results from an AI search query is fine; waiting 3+ minutes is out of the question. Bard’s speed seems tailored to responding to search requests.
- Bard doesn’t appear to be built to perform general tasks. More often than not, it simply throws up its virtual hands and says “Sorry, I can’t do that.” That suggests that it’s not intended to be a general productivity-enhancement tool like ChatGPT. Google may be laser-focused on the search use case, and are thus deliberately avoiding putting time into expanding Bard’s capabilities in other areas.
- Its interface is clean and easy to use. I can easily see it integrate into Google’s search engine as another tab, alongside things like Images and Scholar.
In short, my early testing suggests that Google isn’t trying to compete with OpenAI in building a multi-purpose model. They appear focused on building generative AI into their existing, highly lucrative search business.
Bard is severely limited in its capabilities right now, but the last few months have shown that generative AI models can improve very quickly. As Bard improves, it will almost certainly generate more complete and useful responses.
And if Google directs all its time to integrate those responses into its search engine, it stands a good chance of dominating that space.
I don’t know what’s going on inside Google, but from my testing of Bard, my bet is that they’re willing to cede ground to OpenAI when it comes to general productivity, coding, and other miscellaneous tasks it it allows them to pour all their energy and resources into tuning Bard for search — and to ultimately blow OpenAI and Bing out of the water when it comes to that more specific, limited use case.
I’ve tested thousands of ChatGPT prompts over the last year. As a full-time creator, there are a handful I come back to every day. I compiled them into a free guide, 7 Enormously Useful ChatGPT Prompts For Creators. Grab your copy today!
