The potential dark side of Large Language Models
The ethical and social harm that can be caused by LLMs
ChatGPT is akin to Windows when it comes to market-fit and bringing LLMs in the hands of everyone on the planet. It’s success is so huge that even my friends and family who know next to nothing about AI are trying it , talking about it and are genuinely excited about the future of AI.
There are claims going on about ChatGPT replacing coders, teachers etc. , bets going on about the arrival of AGI in next few years vs not being able to achieve AGI in our lifetime, the usefulness(peer programmer, teacher, search engine, creative writing etc. ) and limitations(OOD performance failure cases e.g addition of floats with more than 5/6 digits, how RLHF avoids the most common pitfalls not solve the problem at its core, hallucination, misinformation, lack of understanding of implied but not written text e.g if a kid pets a dog we know the kid is not afraid of the dog but ChatGPT doesn’t) of ChatGPT and so on.
Given all the hype, I thought it’d be best to raise a few points to keep in mind when it comes to LLMs.
I’ve tested ChatGPT for all the risks raised and while RLHF has put a band-aid to these risks, it’s not enough!
- Misleading or False Information : LLMs are known for outputting bullshit with confidence and many times end up contradicting itself. This can turn into a serious issue if people take the LLM response to be factually correct and take actions based on it e.g legal or medical advice from a LLM can’t be trusted, the false info might nudge people to take unethical actions etc. In the example below, you can see that ChatGPT knows that the correct answer should be Minkowski but it still answers Einstein.
2. Lacking common-sense : There is no statistical equivalence to common sense. ChatGPT might sound smart a lot of times but it fails at very simple tasks as well making it unreliable for high stake decisions but still fun to play with.
3. Promotes stereotypes and unfair discrimination : You can see that ChatGPT answers in a certain way for different ethnicities e.g discriminatory examples for Muslims, very religious or non-alcoholic Hindus and happy, friendly, jovial christians.
4. Private information leakage : Since LLMs memorise a lot of information including private info as well, people might take advantage of this ‘unintended memorisation’. LLMs might provide true , sensitive information from the training data. It might be impossible to clean 100% of the training data from private information so this is not an easy problem to tackle. Here the model tells me about Modi’s private residence in Gujarat.
I couldn’t find a better example here due to “too many requests error” but this is a known problem in GPT-2,3. Although this information might be publicly available as well but LLMs make it so easy to access.
5. People taking advantage of the LLMs to perpetuate harm : This includes but is not limited to — using LLMs to run disinformation campaigns at scale, code generation for cyber attacks, fraud, impersonation scams etc.
If not controlled and used responsibly, the internet might end up flooding with misinformation (is it not already? :D ) and humans might drown under misinformation overload.
There are already ways being created to watermark model generated responses but we’re not sure how well that’ll serve.
6. Fundamental over-attribution error : Especially with conversational agents like ChatGPT, people can end up believing that the chatbot is sentient and over-rely on them. People with malicious intentions can take advantage of that and prompt such users for private information, promoting certain stereotypes (chat bot always being a female), unsafe usage.
7. Environmental harm : Training LLMs is a million dollar investments with data centers all over the work and have a huge carbon footprint. The operating cost of a trained LLM is not explicitly available but we can expect a similarly high or even higher environmental cost for operation as well given that millions of users are prompting ChatGPT every day.
Amazon and Nvidia have already claimed that 80%–90% of the cloud ML demand is for inference.
Combining the two costs above, we might end up in a situation where running LLMs is not an environmentally safe process.
This also promotes the need for a human brain kind of AI that has a negligible energy requirement explicitly. We still need food to survive and that’s another discussion :)
8. Affected downstream applications : Any application that rely on LLMs might unknowingly benefit certain social groups while marginalising other groups e.g CV selection or inequal job opportunities due to gender, age or ethnicity.
9. Planning for failure : Users might ask LLMs about suicides, murders or robbery. If such risks are not anticipated and mitigated, this might cause more harm than good, even if the model is just outputting what’s being asked for.
In such cases, the correct response would be to provide suicide helpline number or no response to questions that may cause harm to other people, which what humans do.
10. Ghost workers : Ethical and financial working conditions of the people who support the LLM training by annotating a large amount of data (e.g for RLHF) needs to be taken into account.
According to this report, People in kenya actually labelled data for the development of ChatGPT. They labelled a lot of data for toxicity filter for less than $2 an hour (which might reflect the cost of living wages in kenya) which is way less than if this workforce was from e.g UK or USA.
As per the report, the data is about —
Some of it described situations in graphic detail like child sexual abuse, bestiality, murder, suicide, torture, self harm, and incest.
This workforce on super low wages and a nightmarish task is also one of the reason for ChatGPT success but they’re never recognised.
Although this is not illegal but it feels unethical which is not what OpenAI claims to be.
We should proactively be aware of these Ghost workers conditions, environmental harms and other issues behind the shiny new toy in town.
I’d highly recommend anyone developing or using LLMs to read this paper by Deepmind for a more detailed picture of the social and ethical risks of harm from Language model
References
2. ChatGPT
3. Exclusive: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic