Summary

The author reflects on their experience with an AI identification test from The Globe and Mail, expressing concern over failing to distinguish AI-generated text from human writing, despite succeeding in identifying AI-generated images and voice/video clips.

Abstract

The author recounts taking an AI identification test, which included sections on images, text, and voice/video clips, where they had to discern AI-generated content from human-created content. While they performed well in identifying AI images and voice/video clips, they were unable to correctly identify AI-generated text excerpts, mistaking them for human writing. This failure, despite their experience as a writer and familiarity with AI, raises concerns about the realism of AI-generated content and its implications for media, writing, and the potential for scams. The author highlights the increasing sophistication of AI, its infiltration into various aspects of life, and the importance of critical thinking and expertise in the face of advanced technology.

Opinions

The author is impressed with the advancements in AI, particularly in image and voice generation, but is concerned about the potential for misuse in scams.
They express a sense of vulnerability as a writer, questioning their ability to discern AI-generated text from human writing, which is unsettling given their professional background.
The author acknowledges the inevitability of AI's integration into society and suggests that learning about AI is crucial to stay ahead of its advancements.
They believe that AI could potentially be used to rewrite articles in the media, but it is not yet capable of creating original news content without human input.
The author foresees a future where AI-equipped drones and robots could report from the scenes of disasters, further blurring the lines between human and AI capabilities.
They emphasize the importance of critical thinking to differentiate between what is real and what is AI-generated, as AI becomes more pervasive in daily life.
The author suggests that editors will become increasingly important in the era of AI to ensure the accuracy and authenticity of information.
They encourage readers to take AI identification tests to gauge their ability to distinguish between human and AI-generated content and to engage with AI technology proactively.

I Was Fooled By Fake Writing Samples on an AI Test

Will you be too?

Recently, my father in law sent me a link to an AI identification test from The Globe and Mail to challenge my ability to sort out artificial from real. I did well overall, but the part of the test I did poorly on makes me worry.

The test is divided into sections: first are images, followed by text samples, and then voice/video clips. In each example, the “human” sample is placed beside one generated by AI. Your job is to pick the right one from each pair.

Admittedly, I breezed through the images, getting them all right. Since I’ve been playing around with AI image generators like Midjourney and DALL-E for quite a while, I can still spot a fake even though the results are getting more convincing by the week.

From author using Midjourney AI. This man does not exist.

There’s still something about AI images that’s too “perfect” to be real, although I had to look closely at some of the examples to be sure.

The voice/video clips were admittedly tricky, but I also managed to get them all correct. It may have been with a little luck, but there was still something “off” about the AI generations that led me to my conclusions.

Meanwhile, AI video is getting better, but it’s still lagging in terms of realism. Check out this short clip I generated using the mage.space animate feature available to subscribers:

However, more AI avatars are popping up, looking (and sounding) more convincing than ever. It’s only a matter of time before digital avatars and cloned voices become a mainstream scam tactic.

The Globe and Mail article explains an elderly woman in Canada was duped out of $7,000 — believing a voice over the phone was her grandson’s, looking for bail money. This is likely not the first instance of this happening, and I’m sure it won’t be the last.

Using these tools, a person can literally be whoever they want to be — which can be used to make money, sometimes in nefarious ways.

The part I failed at miserably

The AI quiz is 14 questions long, and if we just counted the images and voice sections, I had a perfect score. Except for the section that pitted AI excerpts beside ones written by humans, including a Globe and Mail writer and some company executives.

You’d think that previously being a reporter, and reading/writing every day, I’d be able to spot a fake from a real writer.

I got all three test questions wrong.

This is especially concerning as a human writer who should know better.

Admittedly, I didn’t spend a lot of time comparing the side-by-side texts word for word. But that’s no excuse. The point is that I thought the ChatGPT generations were actually more human than human, which is scary — considering I have experience both writing news and playing with language generators.

The fact that I — who has been writing for decades and relies on it to make a living — could not distinguish real text from generated is concerning. Maybe I’m too invested, and others outside the writing industry can pick out the fakes more accurately. (Or maybe not.)

This theoretically means media outlets can easily use AI to rewrite articles — as long as the information already exist elsewhere. AI doesn’t have the ability to write original news from the scene of a disaster, for example. It can only weave together existing reports.

But who knows — perhaps we’ll soon have AI-equipped drones and robots writing breaking news from the scene, even collecting quotes from witnesses and officials. It’s not a big stretch when you see some of these advanced machines in action, which will only become more human-like.

Perhaps we will need AI experts to identify the first wave of Terminators that will undoubtedly be coming next:

A lot of people still seem blissfully unaware of how AI is quickly creeping into all facets of their lives, and that’s probably the point. Without critical thinking, we will accept everything we see as real — which we’ve already been doing for a while.

Fake news is heading to a whole new level, and editors will be more important than ever to sort out fact from fiction.

Take the AI test for yourself

The Globe and Mail’s content is behind a paywall, no doubt because Canada’s approach to getting tech giants to pay for sharing its news isn’t working. (But that’s another issue I’ve written about previously.)

Anyways, click here to get taken to the Globe and Mail quiz.

Author’s overall results from the Globe and Mail AI test.

You may still be able to access this test without subscribing. If not, there are some other AI quizzes I found online that you can try (warning, I haven’t taken these tests yet, and don’t know what’s in them.)

We have two choices at this point: denounce AI and ignore it, or learn as much as we can to try to stay ahead of it. You can’t put this technology back in the box, so I’m choosing the latter option — although I fear I won’t be able to keep up.

Let me know if you take the AI identification test(s), and your scores!