The Handicap of AI: Understanding Why Hands Remain a Challenge for Image Generation

The quality of AI images has astounded the photographic community with its hyper-realism. But it seems that there is one obstacle they keep running into: hands.

Miles Zimmerman, a 31-year-old programmer from San Francisco, was playing around with Midjourney earlier this month and having his mind blown by the AI-powered application that creates visuals from a brief text input.He used ChatGPT to help him make some of his prompts, and one of them was very specific: “A candid photo of some happy 20-something year-olds in 2018 dressed up for a night out, enjoying themselves mid-dance at a house party in some apartment in the city, photographed by Nan Goldin, taken with a Fujifilm Instax Mini 9, flash, candid, natural, spontaneous, youthful, lively, carefree, — ar 3:2.”

Join the Medium Membership Program for only 5$ to continue learning without limits. I’ll receive a small portion of your membership fee if you use the following link, at no extra cost to you.

In a matter of seconds, Midjourney spit out made-up images after made-up images of appealing young people having a good time at a party. He shared the images on Twitter, where they soon went viral.

Services like Midjourney, Steady Diffusion, and DALL-E 2 have seen a huge increase in popularity recently. These apps, which allow anyone to make almost any type of image they desire using simple text instructions and a brand-new form of artificial intelligence known as generative AI, have both generated enthusiasm and criticism.

These programs function because they have been “trained” to identify connections between the countless images that have been downloaded from the internet and the text descriptions that go along with them. Eventually, the program “understands” that, for example, the word “dog” relates to the picture of a canine.

But Why Does AI Struggle?

According to a Stable AI spokesman, one of the reasons AI picture generators struggle with hands is that human extremities are less visible than their faces in the datasets required to train the image synthesizers. Because they are hardly ever seen in huge scale, hands are likewise frequently significantly smaller in the source photographs.

Professor Peter Bentley, a computer scientist and author based in London, claims that the 2D picture producers also find it difficult to understand the 3D geometry of a hand. “Kids understand the fundamental concept of a hand. None of these models fully comprehend what the complete thing is”, he tells the BBC.

While Stable Diffusion released better but still subpar photos, DALL-E produced hilariously poor images. These appeared more plausible while remaining completely incorrect.

Here are some results generated by DALL-E

Amelia Winger-Bearskin, professor of AI and the arts at the University of Florida, explains that generative AI just doesn’t comprehend what a hand is and what its purpose is. She told BuzzFeed, “It merely looks at how hands are depicted in the photographs that it has been trained on. Hands are quite nuanced in photographs. They typically have something in their hands. Maybe perhaps, they’re clinging to someone else”.

Is It Just the AI?

Not only AI, that found it difficult to draw hands; throughout history, artists have also tried to avoid drawing hands due to their complexity. Artists like Leonardo da Vinci didn’t begin observing and drawing hands until the Renaissance.

Winger-Bearskin continues, “Da Vinci was actually pretty concerned with hands and did many, many studies of hands”.

Meanwhile, AI simply observes an image and says, “Well, in this case, there’s only half of a thumb, because the rest of it is concealed behind fabric or clutching onto something, and as a result, when it reproduces it, it’s somewhat deformed.”

My other writings —