Is ChatGPT Killing Medium?

Dealing with a deluge of AI-written articles

I’m an editor of Japonica, a publication on Medium focusing on Japanese life and culture.

We used to receive a handful of articles per week, mostly from Japanese people or non-natives with extensive experience in the country. It usually only took a few sentences to tell if the writers knew what they were talking about.

Occasionally, we’d receive articles that compiled information from Wikipedia, or worse, were plagiarized from other blogs. It was obvious when the first paragraph was written in broken English and the second paragraph had the patina of a professional article polished to perfection.

I wondered why they bothered. Yes, Medium pays for articles, but despite all the articles about making millions on Medium (what happened to all of them?) we know it isn’t very much. And you don’t get hundreds of thousands of views with articles for a niche audience. It hardly seemed worth the effort.

All that changed about 3 months ago with a deluge of people with no clear ties to Japan asking to be added as Japonica writers. I thought that was the result of one of our promotions, or perhaps an article that went viral.

Then their articles started arriving. Most were about some well-known aspect of Japanese culture like manga or sushi, or current topics in the news.

The writing wasn’t bad — in fact, in a way it was quite good. There were no typos or cut-and-paste errors, all the verbs were properly conjugated, tenses aligned, parallel structures properly parallel. All the details we usually have to fix were perfect.

But there were subtle factual errors. I assumed that was because the writers enjoyed aspects of Japanese culture but didn’t have experience living here. So, fine, those were easy enough to fix, too.

But more importantly, the articles were generic. They sounded more like a summary of a Wikipedia page than the sorts of personal stories we typically find on Medium (like this one!). But fine, we’re happy to publish a history of manga or a summary of the Hinamatsuri Festival as long as it’s interesting.

But that was the real problem. This new crop of articles wasn’t interesting. They were full of information, but they were kind of boring.

So we rejected them.

But still they kept coming. More and more every day.

It wasn’t until ChatGPT hit the news last month that we understood what was happening.

People are using ChatGTP to write articles, a dozen at a time, and submitting them to any publication they can find. Instead of taking a few hours per article, with ChatGTP they can have an article in minutes.

Pick a publication and find their topics, and with the snap of the fingers (or the tap of an enter key), presto, an article is ready. No knowledge, no research, no editing required. In fact, it takes no thinking at all.

As editors, should we care? If someone submits an article, does it really matter if they wrote the words themselves or thought of the idea and had the computer write it for them?

I believe it does matter. We’re not just publishing content, we’re publishing people. I want to read your articles and listen to your turns of phrase. I want to hear your thoughts and arguments, even if I disagree with them.

If I just wanted information, I can wade through Wikipedia myself, or log into ChatGPT and have it write exactly what I want.

Unfortunately, it isn’t always easy to tell the difference between AI writing and a generic human article. We run the AI detector provided by OpenAI (the developer of ChatGPT), but most of the time it responds with “unable to determine”. Hey, if the smartest AI in the world can’t tell if its written the article itself, how am I supposed to judge?

We’ve tried a few other detectors and they usually come back with results like 17% human or 98% human. If it’s 98% human, what is the 2% that isn’t? Not really helpful.

So there’s only one solution — publish only good articles by good writers and reject anything that feels generic or boring.

As good as ChatGPT is, for the moment it still has no creativity or voice. It can write a school report better than most students, but at least for now, it can’t write an interesting article.

Maybe in another year or two, AI will be able to write better than writers, the same way it can beat the best chess and Go players.

So at least for the moment, our jobs as writers is safe. But our jobs as editors just became considerably more demanding.

If you like my articles, you’ll love my mystery novel, To Kill a Unicorn, about the first startup to monetize murder.