Assessing the Risk of AI Models: The Problem with Double-Standards
Generative AI models are opaque. But then again so is human behavior. Why are we demanding accountability for AI and not for humans?
Note: If you are not a Medium member, you can read this story for free using this Friend Link. I encourage you to join our community!
With the release of OpenAI’s ChatGPT, my sense of awe has not diminished after several months of daily use. Once again, I had to learn how to respond to the prompt — this time, GPT’s.
My jaw-dropping encounters with GPT remind me of a cartoon by Sidney Harris that I saw several years ago. A young ambitious scientist is presenting to a dowdy elder scientist a chalkboard filled with math calculations. The left part of the board has a complicated set of mathematical formulae and to the right, another set. Connecting the two sets are arrows pointing to and from the words, “THEN A MIRACLE OCCURS.” The elder ponders this conclusion and says to the younger: “I think you should be more specific here in step two.”
While I have felt like a moth drawn to a bright GPT light, others have expressed fears of being drawn into a fatal GPT flame. I suppose when a MIRACLE OCCURS to me, a calamity occurs for others. There appear to be two warning flags waving in front of generative AI models.
The first is the intentional misrepresentation of generated content by humans for the purpose of sick humor or sabotage. That flag points to real danger in real time. Recently, an AI-altered photo was posted showing the Pentagon in flames, which caused the financial markets to hiccup until the ruse was identified. Governments around the world are responding to threats like these, as they should be.
The second red flag warning about the dangers of GPT models concerns the explainability of its generative products. Based on artificial neural networks (ANN), transformer AI models are computationally opaque. No one can explain, by any method of logic, how the command input to GPT becomes its response output. Suddenly, rational thought, reflecting deductive logic, has given way to nonlinear thought, reflecting black box unaccountability. In the absence of logic, mistrust rises. In the presence of uncertainty, rules are tightened.
Medium contributors are addressing this mistrust. In a recent posting (The Real Problem With Artificial Intelligence), Will Lockett wrote this about GPT-4:
While it is very, very good at mimicking natural human writing, it doesn’t have the logic to construct a cohesive argument. It makes up false facts, and its line of reasoning can be incredibly flawed.
Lockett references Geoffrey Hinton who quit his job at Google to warn of the dangers of generative AI. One of Hinton’s fears is that “these things are getting smarter than us.” In his Medium essay, Lockett disagrees with Hinton, stating: “AI has something called the black box problem, which makes them far from intelligent.”
Samir Rawashdeh, Associate Professor of Electrical and Computer Engineering at U Michigan-Dearborn, characterizes the black box problem as knowing something without being able to account for how you learned that something. “It’s not that you forgot. It’s that you’ve lost track of which inputs taught you what and all you’re left with is the judgments.”
Another AI researcher and author, Kate Crawford, who holds positions at USC and Microsoft, put the black box problem this way:
(AI) is not intelligent in any kind of human intelligence way. It’s not able to discern things without extensive human training, and it has a completely different statistical logic for how meaning is made. Since the very beginning of AI back in 1956, we’ve made this terrible error, a sort of original sin of the field, to believe that minds are like computers and vice versa. We assume these things are an analog to human intelligence, and nothing could be further from the truth.
To a point, I understand the fears of some AI architects and users that the generative output from a deep learning AI model, like GPT-4, should be suspect. Deep learning models are built from many terabytes of information scraped from every node on the Internet, which are lumped into large language datasets where information bits are sorted and tagged for associations. Digital representations of this information, which is sometimes valid and sometimes blemished, are transformed into virtual representations of knowledge.
Can algorithmic knowledge be trusted? This is no small question when the validity of the generated knowledge is used to drive healthcare planning, medical treatments, financial planning, political candidate voting preferences, meal planning for someone with allergies, or scholarly research.
The black box problem of AI “explainability” (XAI) is real. The distrust arising from AI’s opaque reasoning is real. The question then becomes: can future models of AI be built for greater accountability? I am not a computer engineer so I cannot speak to the explainability of future AI models. However, I am a psychologist and I have some views about the above-quoted statement by Kate Crawford:
Since the very beginning of AI back in 1956, we’ve made this terrible error,…to believe that minds are like computers and vice versa. We assume these things are an analog to human intelligence, and nothing could be further from the truth.
I beg to differ with Kate Crawford. In a series of essays that I will post to Medium in separate installments, I will explore the popular but mistaken belief that humans engage in rational, linear, accountable thought. I will argue that the cognitive processes of humans are not so different in outcome from those of deep learning machines that engage in mostly nonlinear, “opaque” thought.
Human and generative AI use different processes to transform input into output, but at certain levels of abstraction, these processes are eerily similar. If the output of generative AI seems spooky, human behavior is often no less so.
As an example of opaque and nonlinear human behavior, I present to you a case study that claws at the myth of humans engaging in logical and auditable thought toward self-optimizing ends, thereby demonstrating rational behavior.
Dr. Jacquelyn Starer is a licensed psychiatrist who specializes in child and adolescent psychiatry, practicing for over 20 years in three Northeastern states. She is also a licensed gynecologist. As an active member of the medical community, she served as the director of pediatric emergency medicine at NYU Langone Hospital-Brooklyn and as an assistant professor at NYU Grossman School of Medicine. Most recently, living in Ashland, MA, people know her as Jackie, the 68-year-old who drives a sleek convertible, adores ballroom dance, has advocated for safer sidewalks, and has recruited parents to staff the refreshments booth at a school fund-raiser.
Professionally, colleagues knew Dr. Starer as the president of the Massachusetts Society of Addiction Medicine, consultant to the Massachusetts Department of Public Health, and a double board-certified attending physician at Brigham and Women’s Faulkner Hospital. “Jackie was a caring person who took care of lots and lots of people and did a lot of good in the world,” said Dr. Alan Wartenberg, who completed his fellowship at Brigham and Women’s Faulkner Hospital alongside Starer in the 1990s.
Given the stature of this respected professional physician, how do we explain that, on January 6, 2021 at the U.S. Capitol, police body cameras allegedly show Dr. Starer balling up her fist and punching a female officer on the side of her head?
Because of her documented participation in the attack on the Capitol, Dr. Starer is facing federal charges, including violent entry and disorderly conduct on Capitol grounds, entering a restricted building without lawful authority, and engaging in physical violence in restricted buildings or grounds. As of the writing, there are no press reports regarding the status of these charges against Dr. Starer, although it appears her medical license is under review in New Jersey due to alleged ethics violations.
A British psychiatrist, Paul Plsek, once mused that baking a cake is simple, landing an astronaut on the moon is complicated, and raising a child is complex. From a different perspective, baking a cake is chemistry, landing a person on the moon is physics, and raising a child is biology, which is by definition complex.
Given that human behavior, with its biological foundation, is complex, by what protocol would we conduct a rule-based audit on the mens rea of Dr. Starer that led her to join a mob of rioters at the Capitol on January 6, 2021?
In this age when AI models approach artificial general intelligence (AGI), a timely question is this: What is rational man? Surely a rational person can answer questions about decisions they have made and actions they have taken. After all, that’s what it means to be rational.
But how, for example, do we conduct a rule-based audit on a 20-year-old youth who invades a primary school in New Town, CT, and kills 20 children between the ages of 6–7 and six adults? As a nation, how do we logically account for the occurrence of 380 school shootings since the first Columbine High School massacre in 1999? How do we use logic to explain the post-Columbine deaths of 191 children and adults at 354 school campuses, traumatizing over 331,000 young people exposed to gun violence? How do we account for 257 million voting-age Americans owning 434 million guns?
Human behavior is complex. Aristotelean logic, with its syllogistic reasoning, will not unfold our opaque behaviors. The median age of a school shooter is 16. Most of these shooters exhibited no mental illness and many use planning processes that could be considered rational were it not for the unspeakable outcomes. A popular writer, Steven Pinker, argues in his recent book, Rationality: What It Is, Why it Seems Scarce, Why It Matters (2021): “Rationality ought to be the lodestar for everything we think and do. (If you disagree, are your objections rational?”
Dr. Pinker, I do object. Perhaps I betray my irrationality (as evidenced in Pinker’s thinking any objection is irrational) when I express my firm belief that the concept of rational man is dead. We just haven’t written its obituary. I’ll address this more in subsequent essays.
M. Beatrice Fazi, an AI researcher at Sussex University, quotes a scholar who warned that opening a black box could result in finding the box to be empty. Fazi finds the greater risk is realizing that “…there is nothing to translate or to render precisely because the possibility of human representation never existed in the first place.”
In future essays, I will present a framework for explaining why both human behavior and generative AI models are opaque, albeit for different reasons. I will question why generative AI models are being judged by standards of rationality when human behavior cannot meet that standard.
My comparative explanatory framework will be based on the principles of dynamical systems theory, otherwise known as complexity science. My contribution will be that of an argued opinion, not a proof. My goal will be to show that there is not as much explainability distance between the output of human behavior and generative AI as some would have us believe. I will argue in favor of rule-based behavior for both humans and machines, but just not a rule model based on the traditional logic formulated 2300 years ago.
Generative AI models are both a bright light for social good and a forest fire of social risk. The opacity of future AI models will need to be addressed in order to unleash its potential for social good. But we should not attempt to establish lucidity in machines using standards that are based on outdated models of human behavior.
To quote Rachel Maddow, “Watch this space.”
References
Fazi, M. B. (2020). Beyond Human: Deep Learning, Explainability and Representation. Theory, Culture & Society, 38(7–8), 55–77. doi:10.1177/0263276420966386
Pinker, S. (2021). Rationality: What It Is, Why it Seems Scarce, Why It Matters.
Plsek, P. E. (2003). Complexity and the Adoption of Innovation in Health Care. Paper presented at the Conference on Accelerating Quality Improvement in Health Care: Strategies to Speed the Diffusion of Evidence-Based Innovations, Washington, DC.
The Real Problem With Artificial Intelligence
Even experts working on the technology misunderstand AI
