When AI Prefers Caucasian

Spotting and mitigating racial bias in AI creations

Generative AI has reached a point where it can create promotional scripts and convert them into videos in less than five minutes. I know because I’ve done this with ChatGPT+ and achieved successful results. But with this amazing capability, it’s possible for AI to weave in unexpected biases. I also know this because I witnessed it firsthand.

Initially, my goal was simple: to create a promotional video to drive registration for an upcoming webinar. But as I delved deeper, I found myself on an unplanned detour, exploring the hidden biases in AI that often go unnoticed. It was a journey that started with a simple script and ended with eye-opening insights into the intricacies of AI.

The journey illuminates how even routine tasks should be reviewed with curiosity to expose subtle biases and identify methods to mitigate their influence in our interactions with AI.

“In one test, I specified ‘a middle-aged man in a business suit,’ deliberately omitting skin tone to see if the AI would prompt for this information. It didn’t.”

Recognizing Bias with ChatGPT

Recently, I challenged ChatGPT to draft a script for an upcoming webinar and then animate it with a photorealistic human avatar. Since creating a video isn’t something ChatGPT can do on its own, I turned to a plugin for this — specifically, one from HeyGen.

This plugin makes it possible for ChatGPT to step beyond text, bringing scripts to life through video. With HeyGen, I could craft and visualize a script, turning words into an engaging video with an AI-generated avatar. While the end product was useful for my needs, I stumbled upon something that I had not anticipated: a nuanced understanding of bias in AI.

An AI Video Sparks Curiosity

After approving the script, my final instruction to the AI was clear: ‘Create a video with a professional spokesperson using script 1, using emphasis and excitement to increase attendance.’ The result was a polished video, but something caught my eye — a subtle but significant alteration in ChatGPT’s response.

It had changed ‘professional spokesperson’ to ‘professional spokeswoman.’

This deviation piqued my curiosity, prompting a deeper dive into the mechanics behind the scenes. Digging into the API call used for generating the video, I uncovered predefined attributes for ‘gender’ and ‘skin tone.’ Both of these categories were already filled in, a detail that wasn’t initially apparent.

This discovery led to a series of probing questions:

• Why hadn’t the AI asked for my input on these attributes when I requested the video?
• Was this a default setting embedded within the application, or was it a reflection of ChatGPT’s underlying programming?

These questions were more than mere technical curiosities; they hinted at the nuanced ways biases could be woven into AI systems, often subtly and without explicit user guidance. As I delved into these questions, I realized that the choices made by the AI — seemingly mundane and automated — carried implications far beyond simple video production.

The decision to default to a ‘spokeswoman’ and the preselection of skin tone were not just technical choices but reflective of underlying assumptions programmed into the system. These defaults, while perhaps intended to streamline the process, raised important concerns about representation and decision-making in AI.

Seeking to Understand the Reasons for Bias

To unravel the reasons behind this, I embarked on an experiment. I asked ChatGPT to regenerate the video, but this time, I included specific instructions for the AI to ask me any clarifying questions that might assist in the task. This approach, which I often use in my prompts, is like a black belt technique in generative AI: it invites the AI to engage more deeply in the task, seeking to understand the nuances of the request.

Sure enough, ChatGPT’s response was different this time. It asked for details about the spokesperson’s appearance, including gender, age, and attire — aspects it had previously assumed or defaulted to. This shift in interaction was revealing. It highlighted how the specific framing of a prompt could significantly alter the AI’s approach to a task, shedding light on the importance of human input in guiding AI outputs.

Curiously absent still was any clarification for skin tone or race.

In the experiment, ChatGPT asked for additional details to ensure the quality of the video. Despite these queries, it notably omitted any clarification regarding skin tone or race. So, when asked to describe the spokesperson, I specified ‘a middle-aged man in a business suit,’ deliberately omitting skin tone to see if the AI would prompt for this information. It didn’t.

ChatGPT then proceeded to call the plugin, inputting ‘male’ and ‘light’ for gender and skin tone, even though these specifics weren’t provided by me. This phase of the experiment highlighted a peculiar aspect of the AI’s decision-making process. The choices made for ‘male’ and ‘light’ were seemingly autonomous, yet they were not completely aligned with my input.

Things were starting to get weird. First, if this guy, who is straight out of central casting for the next Twilight sequel, is supposed to pass for “middle aged,” then I’m ancient!

Despite specifying age and attire — as requested by ChatGPT--these details were not relayed to the plugin, revealing a disconnect between the input provided and the AI’s interpretation and execution. I wondered if ChatGPT was hallucinating about available parameters.

I continued down the rabbit hole and asked ChatGPT about the available input parameters for the plugin. ChatGPT’s response outlined that the primary input was ‘text’, with ‘title’, ‘gender’, and ‘skin tone’ as optional parameters. It also provided the options for each category.

Notably, for ‘skin tone’, the choices were limited to ‘light’, ‘dark’, or ‘asian’.

Armed with this information, I questioned ChatGPT about why it asked for details like age, which it couldn’t use in the video, and why it omitted to ask about skin tone. ChatGPT’s explanation was somewhat vague, suggesting it was trying to grasp my overall vision, even though some of this information was irrelevant for the plugin. Interestingly, it also apologized for not inquiring about skin tone, acknowledging this as a critical detail with the potential for introducing bias.

It almost seemed like ChatGPT was offering an apology for something beyond its control. This made me wonder: was the limitation rooted in the plugin’s API, leaving ChatGPT as the front-facing entity left holding the bag?

More insight, and more hallucinations

After ChatGPT had received more detailed input from me, I was eager to see if its approach to creating the video would shift. Intriguingly, it did inquire about skin tone this time. However, it also introduced two irrelevant factors: pacing and emphasis, attributes that the plugin couldn’t actually process. It appeared ChatGPT was keen on understanding my artistic vision, even delving into aspects that were technically infeasible.

There was no need to continue traversing down the hallucinatory rabbit hole with ChatGPT. Having conducted these tests, I gathered a wealth of insights and uncovered key insights into how AI interprets and executes tasks.

More importantly, the exercise shed light on the unintentional introduction of bias, a critical aspect often overshadowed by the technical prowess of AI systems. With these learnings in hand, it was time to step back and analyze what had transpired.

Is AI to Blame?

Understanding the nature of the video creation plugin is crucial here. This tool from HeyGen isn’t really a generative AI platform for crafting new, lifelike avatars from user prompts. Rather, it’s equipped with a small selection of virtual humans designed to mimic script delivery through facial expressions and body movements. The limited portfolio of these pre-existing virtual characters — just three in the free version accessible to ChatGPT plugins — is a significant constraint.

Additionally, my observations have shown that ChatGPT often aims for swift, pleasing responses to users, aligning with what it calculates as the most probable correct answer.

Take, for example, the scenario of asking the chatbot for a lasagna recipe. ChatGPT typically wouldn’t ask whether you prefer a vegetarian option unless prompted. It’s on the user to specify such preferences upfront or to ask ChatGPT if it “has any other questions.”

In the context of video creation, proactiveness from ChatGPT in asking about gender or skin tone would have been beneficial. In this instance, it defaulted to the plugin developer’s preset parameters: ‘female’ and ‘light.’” However, ChatGPT doesn’t inherently grasp the nuanced sensitivities or potential biases that omitting such queries can introduce. In fact, that’s exactly what it told me when I asked it to reflect on the learnings from this experiment.

How Can We Avoid Bias in AI Systems?

This incident brings to the forefront a crucial element of AI development: the unintentional embedding of creator biases into AI systems. These biases might stem from the training data, the unconscious preferences of developers, or the inherent limitations of the technology. The consequences are far-reaching, underscoring the need for heightened transparency and greater control for users over AI-generated content. The goal should be to achieve outputs that are not only efficient but also fair and inclusive.

Default settings are crucial in applications, as they streamline execution flow. Perhaps these parameters should have been mandatory user inputs, rather than optional. This improvement is something for the plugin developers to consider.

Similarly, it would be beneficial if the applications calling these plugins, like ChatGPT, were programmed to prompt for more detailed input based on the parameters. This is an area where ChatGPT and the wider development community can evolve.

And for our part as users? Perhaps we need to be more descriptive in our requests, to always check calling function codes, and to maintain a healthy curiosity as to how bias can be introduced. That improvement rests on us, humans, to improve.