Would You Pass An AI Detector Test? Don’t Be Too Sure, Lol
Do yourself a favor. Test your writing. But use the crappy test that never gets it right. It will make you a better writer, I promise.
Y’all, sometimes the world is too stupid for words.
When ChatGPT first launched, lots of people worried about the impact on writers. Omg, it’s coming for our jobs!! Places like Buzzfeed fueled the fears when they started using ChatGPT to create content.
Turns out, that’s not the biggest problem. Oh hells bells, no.
As it turns out, the real problem is way stupider.
Brace yourself.
Have you ever put your own writing into an AI detector? Until this week, I hadn’t. Why would I? I know I wrote it, I don’t need to check to see if I used ChatGPT. lol. How dumb. I know I didn’t use AI.
But you know what?
That overconfident attitude is getting people in big, big trouble.
This week, I ran across a bunch of freelance writers talking about clients running their writing through AI detection and refusing to pay them. The writers swore up and down they did not use AI.
But still. They failed the test. They’re not getting paid. Some got fired.
Wow. Ouch. That sucks.
Writers are getting accused of using ChatGPT to write articles. Except? They didn’t.
It started with one writer posting it on Twitter, but a whole bunch more piled on saying they’ve had the same thing happen.
A bunch more people piled on and said they pasted their own writing into AI detectors and failed. Like this guy… and a bunch more like him. There’s also a ton of similar threads on Reddit and Upwork.
Good grief, I can’t imagine doing the sweat and research to write a 500 or 1200 word blog post only to have it fail AI detection. And then having someone refuse to pay for your work.
Because nuh-uh, you failed the test.
I read a painfully long thread on reddit where the guy pleaded with his client to tell him which “detector” he used so he can see why he’s failing. The client said nope. He’s holding the writer’s money hostage.
Wtf? That’s not okay.
It’s not just freelancers. It’s everywhere we write.
Some blogs that accept guest posts are now running guest post submissions through AI detection tools, too. Wow.
I found a couple of prominent blogs who caution writers that they run guest submissions through AI detection. I won’t name them because my pockets aren’t deep enough to get my butt sued for spilling the beans.
So I grabbed some of their “staff” pieces. Ran them through AI detection. Some of them failed. Oops.
There’s a sick humor in some guy saying he’ll use AI detection on guest posts, while his own writing scores 45% AI, don’t you think?
He probably doesn’t even know.
Like, how would he? Why would he run his own writing through an AI detector? He knows he wrote it. It’s *you* he’s not sure sure about. lol.
It’s happening at Google, too
Also? Google says AI content is against their guidelines, so they’ll find and devalue pages that have AI content. Oops. Fell off page one.
Go ahead. Count the ways that can whack us. Doesn’t matter if you’re writing posts on your own website or some writing site, if your writing fails an AI detector test, you’re going to get whacked.
The dead stupid irony of AI detection
There’s a real irony to using AI detection tools in the first place. Because AI was trained on us. Literally. They fed it giant swaths of the internet to teach it how to write. Wikipedia, Gutenberg, Common Crawl, etc.
Which basically means AI learned to write from us.
Now if we write like ourselves, we fail the dreaded AI test.
It would be like training a robot to write just like Stephen King or Margaret Atwood and then, when the robot can emulate them sufficiently well, we turn around and critize them for writing like a robot.
What a stupid world we’ve built.
Me vs. ChatGPT, the copy…
I decided to duke it out. But first, I needed some copy. So I grabbed one of my drafts.
Get ready to be stunned. Here’s ChatGPT’s writing;
I asked ChatGPT to write similar copy. It’s not bad. Wow. I’ve seen worse.
Next, I plugged both of those into AI detectors.
Here’s the AI detection test results…
To “pass,” the AI detection tests only need to get 2 things right.
- They need to know my text is human
- They need to recognize ChatGPT as AI.
1. GPT Zero failed to detect AI
GPTzero.me knew mine was human, but thought ChatGPT was human, too. — SCORE: F [FAIL]
2. Writer.com: failed to detect AI at all
The writer.com ai detector said both were entirely human written. — SCORE: F [FAIL]
3. What a joke, OpenAI can’t recognize it’s own work
OpenAI created ChatGPT, but it’s own AI detector failed, too. — SCORE: F [FAIL]
4. SEO.ai could cause me problems
SEO.ai said ChatGPT was 91% AI, but also said my writing was 26% AI. — SCORE: F [FAIL]
5. Content At Scale AI Detector did a little better
The Content At Scale AI Detector knew my writing was human, but thought ChatGPT was a combination of human and AI. Nope. Sorry. Wrong. — SCORE: F [FAIL]
6. CopyLeaks.com was the only test that got it right
Copyleaks nailed it. It was the only test that knew human from AI. — SCORE: A [is for awesome!]
Ever heard of perplexity and burstiness?
One of the tests that failed taught me something super interesting. The AI detectors didn’t think my writing was AI . It was mostly the opposite. They though ChatGPT was human. But one of the tests that failed had some interesting stats at the bottom. Here, look…
See the stats for perplexity and burstiness? Let’s talk!
When I saw perplexity and burstiness in the test results, I dug in to learn more. It’s pretty cool.
Here’s the thing. AI can’t actually write. Not like you and I do. They’re not human. They can’t think and come up with ideas.
You know how AI tools like ChatGPT and Jasper.ai work?
They’re probablity engines. They were “fed” content from the internet. So when you type in a prompt, they do math to determine the “odds” of any given word coming next. That’s how they write. By doing math.
What are the odds of this word appearing next? That’s how they write. It’s math, not writing. And they do it blindingly fast. I wrote an in-depth post about how ChatGPT works if you’re interested.
Perplexity is a robot saying oh, wtf?
When I use phrases like “got stupidly rich” or “banged the book out in 3 months flat” — it makes the AI detector go cross eyed. Basically, it’s perplexed at how I strung words together.
Perplexity is when you put words together in ways robots don’t expect because the probability of those words occurring next to each other is low. My perplexity score was 108. ChatGPT’s was 43.
That’s the difference between a human and a robot.
ChatGPT doesn’t string words together in strange ways.
Burstiness is the same, but for sentence cadence.
If you take a hard look at ChatGPT text, there’s not much variability in sentence length, rhythm and cadence. You know?
Here’s a thing humans do.
Sometimes we write a great big long sentence because that’s what we need to get the thought across because it’s complicated and nuanced. But then? A shorter one. And one somewhere in the middle, because that’s just how we humans talk. There’s wide variety in sentence length. See?
Humans do that. Robots don’t.
That’s why my burstiness was more than twice the score of ChatGPT.
So why are people failing Ai detection tests?
It would be easy to point the finger at writers and tell them if they’re failing AI detection, they need to write better. Maybe some do. But it’s not the whole story. Not by a long shot.
In one of the reddit threads, one writer was at wit’s end. He said the client kept telling him he needs to be more personable. More conversational.
He said look, I get it. I’m trying. But I also need to cram in a whole bunch of keywords, use technical phrases that aren’t friendly or personable, and I need to make sure the post is optimized for Google and tests well in SEO tools. It’s a tall order and he was super frustrated.
Let’s not forget that copywriters also need to do the research before they write a piece. You can’t write 1200 words without research.
Let’s not forget revisions. There’s always revisions.
How much do you think he was getting paid?
Often, the clients want the sun, moon and stars, but don’t pay accordingly. According to copy.ai the average price freelance writers can expect to get paid is 10 cents per word, give or take.
Too often, freelancers have to crank out articles really fast. Because they need to get two more articles out today to make the rent. When you have to churn out content at a ridiculous pace, it’s too easy for the writing to get formulaic. Before ChatGPT, it didn’t matter. Now it does.
The solution? Pay them better.
I used to do SEO copywriting. I charged $1/word and that was a decade ago. What they’re asking for is a tall order. Ten cents a word is crap.
Do yourself a favor. Test your writing. It will make you a better writer, I promise.
But don’t use the AI detector test that works. Go use GPT Zero. It can’t seem to tell AI from human, but what it *can* do is make you a stronger writer.
After you paste your story in, scroll to the bottom and look at the perplexity and burstiness score. The higher those numbers are, the more engaging your writing will be. Trust me, you’ll appreciate the results.
Happy writing!
