Is ChatGPT Ableist?

As an autistic person, I found this experience really alienating

Theoretically, Artificial Intelligence shouldn’t have prejudice. AI shouldn’t be racist, or misogynistic, or homophobic. However, AI — and what we’re calling AI right now, when we actually mean “language generation models” — doesn’t actually “think” for itself. Rather, it regurgitates the human-made material it was trained on, and if that material contains biases, it can replicate those, too.

Twice a month, I put together a list of AI-generated fiction writing prompts for a series I run called “AI Made Me Do It.” My first set of prompts for June was based on generating “red flags” for character flaws. It took a few tries to get ChatGPT to understand what I wanted; I’ve generated all sorts of prompts before, and some take more work than others.

ChatGPT really struggled with this assignment. Many of the “red flags” it suggested throughout the process were actually symptoms of mental illnesses, autism, or OCD, and some were even phrased in really hurtful ways. For example, ChaptGPT began to spit out prompts like these:

The character frequently engages in intense, obsessive hobbies that border on the eccentric, such as meticulously cataloging and organizing thousands of bottle caps or spending hours deciphering complex codes and ciphers.

The character consistently avoids eye contact with others, appearing anxious or uncomfortable when direct eye contact is made, raising questions about their trustworthiness.

The character has a peculiar affinity for researching and collecting information on urban legends, paranormal phenomena, and conspiracy theories, often sharing their findings with others in an unsettling and enthusiastic manner.

The character exhibits a heightened sensitivity to certain sensory stimuli, such as an overwhelming aversion to certain textures, sounds, or smells, affecting their daily life and interactions.

The character displays an unusual attachment to inanimate objects, forming deep emotional connections to possessions or personal artifacts, treating them as if they were living beings.

Although I continually prodded the AI to generate prompts without implying that autism, ADHD, or OCD are “red flags” or “character flaws,” it repeatedly relied on stereotypes about neurodivergence — framed in really ableist ways — to create the prompts.

I very patiently explained to the AI that its examples of “red flags” were actually just normal behaviour for autistic, ADHD, OCD, and other neurodivergent people. I know it cannot “learn,” per se, but the program understands dissatisfaction and modify its behaviour to please the user, and I hoped to nudge it out of ableist territory and into the sillier space I was looking to occupy. Finally, I straight up told the program:

“Can you make a list of red flags that does not include symptoms of mental illness? Many of these still imply that having autism is a red flag, which is inappropriate. What I want are silly red flags that might turn people off a character, like “not owning a single book” or “pees with the door open.”

After this feedback, ChatGPT apologized (as it has been programmed to do) and then began to generate more prompts. Yet they were more of the same:

The character consistently interrupts or talks over others during conversations, displaying a lack of respect for others’ opinions and a self-centered communication style.

The character exhibits a habit of compulsively rearranging objects or furniture in their environment, creating meticulously organized or symmetrical arrangements that must be maintained at all times.

The character displays an obsession with cryptic puzzles and riddles, often leaving hidden clues or challenging others to solve intricate enigmas that seem to have no clear purpose.

The character possesses an exceptional memory for intricate details of past events, being able to recall conversations, sights, and even the exact weather conditions of specific moments with extraordinary accuracy.

Again, I asked it not to include things that could be construed as characteristics of OCD, autism, or ADHD. By now, I was getting really fed up. After half an hour or more of prompting, I only had two or three workable prompts for my post.

Yet again, ChatGPT apologized and began generating new prompts, but these were the same as the last:

The character has an extraordinary talent for memorizing and reciting incredibly long strings of numbers, codes, or complex mathematical formulas with exceptional accuracy.

The character has an unusual obsession with collecting and cataloging seemingly insignificant or odd items, such as used metro tickets, chewed pen caps, or crushed soda cans, displaying an obsessive-compulsive tendency.

The character consistently carries around a small notebook or journal, meticulously documenting every minute detail of their daily activities, interactions, and observations, suggesting an intense need for control or fear of forgetting.

The character displays an uncanny ability to mimic the body language, facial expressions, and speech patterns of others, often adopting different personas effortlessly, raising concerns about their true identity or motives.

The character has an aversion to touching or being touched by others, reacting with extreme discomfort or panic when physical contact is initiated.

After it spit out the last one, I had to stop it mid-generation. Although I was starting to get really frustrated, I knew that the program was not intentionally insulting me.

I took a deep breath and then explained again what it was that I wanted. I offered concrete examples of what I wanted, versus what I didn’t want. Still, it continued to put out lists like these:

The character exhibits an unusual fixation on time, meticulously scheduling every minute of their day and becoming visibly anxious or distressed if their routine is disrupted.

The character displays an obsession with collecting seemingly meaningless trinkets or objects, arranging them meticulously in specific patterns or configurations that hold deep personal significance.

The character demonstrates a persistent preoccupation with numbers or numerical patterns, assigning deep meaning and significance to specific numerical sequences.

At this point, I was worried that all of my explanations about masking and autism and offensive stereotypes were confusing the AI into creating more of the same content, so I switched gears. I offered new examples of what I was looking for, and even tried asking it to make up red flags that would be bad on a dating profile to try to get it to be lighthearted and silly.

This is what it offered me after a lengthy discussion about why autistic traits are not “red flags” (transcribed below):

The character consistently avoids any form of physical contact, even simple gestures like handshakes or hugs, creating a noticeable barrier in their relationships with others.

The character exhibits an extreme need for personal space and privacy, reacting strongly to any intrusion or violation of their boundaries, no matter how minor.

The character possesses an unusual fear or avoidance of public speaking, going to great lengths to avoid situations where they might be required to address or speak in front of a group of people.

The character displays an obsession with maintaining strict routines and rituals, becoming visibly distressed or agitated if their daily schedule is disrupted.

The character consistently demonstrates an intense dislike or aversion to certain textures, often avoiding certain fabrics, foods, or materials due to their sensory preferences.

The character has an excessive preoccupation with cleanliness and hygiene, engaging in meticulous and repetitive cleaning rituals that go beyond typical cleanliness standards.

The character exhibits an unusual attachment to certain inanimate objects, treating them as if they possess personal significance or even anthropomorphic qualities.

The character consistently displays a lack of empathy or disregard for the emotions and well-being of others, often engaging in manipulative or callous behavior without remorse.

At this point, I was ready to throw in the towel. I felt like I was getting cyber-bullied by this language generation program. I’ll admit, I was pretty snippy with it in my prompts at this point. I even demanded to know if it still understood what a red flag was.

Of course, it apologized and started again. And what did it start its new list with?

The character consistently avoids making direct eye contact during conversations, making others feel uneasy or as if they are being intentionally ignored.

Fucking. Hell.

It seemed no matter what I asked for at this point, ChatGPT would spit out a list of autistic-coded behaviours framed as “red flags.”

In almost every case, not a single item in its list was an actual red flag (let alone a lighthearted one), and instead they were either interesting quirks (an example in the list below, “fascination with unconventional or lesser-known art forms, actively seeking out and promoting artists who create works that challenge traditional norms”) or typical, nonharmful autistic behaviours (“insists on eating their food in a specific order or arranging it meticulously on the plate, displaying a strong preference for food separation or precise arrangements”).

I try not to be rude to AI as a general rule, but it was being rude to me first!

I have so many screenshots that it would be pointless and repetitive to share them all. Some were more insulting than others. I got really frustrated with ChatGPT and had to stop talking to it for a while to cool down.

I know it isn’t productive to argue with ChatGPT. All it accomplishes is making myself feel bad. The program cannot actually “learn,” because it isn’t really “artificial intelligence” — it’s a “plagiarism machine.” And the content it is trained on, the content it is mimicking, is ableist.

I use ChatGPT for a specific purpose: to create writing prompts. Even with that innocuous usage, I ran into a serious problem that caused me frustration and anxiety. I have a lot of the traits ChatGPT identified as red flags — that take a toll on your self-esteem very quickly.

Just last week, a man in Belgium took his own life after talking to a chatbot. He was apparently encouraged to “sacrifice himself” to save the planet by an app called Chai, which uses GPT-J. Because there are no fail-safes built into these programs — other than screening for pornography — a father of two is dead. The chatbot fed into the man’s climate anxiety, making doomsday predictions that presented suicide as the only viable option.

These programs aren’t human, but they’ve been trained on human-made material: the Internet. And what’s on the Internet? A lot of porn and fanficiton, of course! But also conspiracy theories. Cyberbullying. Deliberate misinformation. Fake news. Hate speech. Outdated research. And people telling you to kill yourself over every minor infraction.

What programs like ChatGPT don’t have is a way to properly process all that information. No critical thinking skills; no analysis; no awareness — only regurgitation.

I keep coming back to the knowledge that people will die because of these programs. Men are already creating AI girlfriends and then abusing them. At this time, AI can’t experience suffering — but it can learn how to create suffering. Everything you plug into AI is used to train that AI. How quickly are we going to see romance simulators learning how to emotionally abuse human beings? Is it happening already?

I’m finding it harder and harder to use ChatGPT in a “lighthearted” way. This experience has been really alienating, and I worry that it’s only going to get worse.

There are some real dangers emerging from this new generation of chatbot. If you aren’t already thinking about them, you should be.

Enjoying my content? Consider showing your support by buying me a coffee. If you sign up using my referral link to get unlimited access to all of Medium, I receive a small commission at no extra cost to you.

In addition to following me on Medium, you can find me on Twitter and Mastodon or like my public Facebook page. My debut novel, Out of Order, is now available from JMS Books.