Can Generative AI Create Convincing 3D Renders? A Real-World Test

Comparing Midjourney to three professional 3D designers

For many creative fields, generative AI is the ten-ton elephant in the room. Between the release of ChatGPT and GPT-4 from OpenAI, the launch of Midjourney 5, and Google’s entry into the space with their new Bard service, it feels like generative AI is moving at a break-neck pace.

What does this mean for 3D designers? Can an AI system create 3D renders without the need for human designers, HDRi Maps , DCCs, and the other tools of the 3D design trade?

My client, CGI.Backgrounds, recently decided to find out. CGI.Backgrounds produces high-quality HDRi Maps for the 3D design community. Their maps provide the backgrounds and lighting data for virtual productions — TV commercials or even full movies shot in fully-virtual environments.

The company works with a wide community of professional 3D designers. To test how their work compares to the creations of an AI system, we took several 3D renders created by the CGI.Backgrounds community of professional designers and then asked the AI image generation tool Midjourney to generate similar images.

Crucially, we only gave Midjourney text-based prompts for each image–the system didn’t see any of our HDRi Maps or the completed renders.

Here are our results–and why CGI.Backgrounds believe that 3D designers have the advantage over AI in its current form.

Automotive Renders with Midjourney — the Dodge Challenger

Allan Portilho, one of the foremost automotive rendering experts and a prolific 3D designer, created this stunning 3D render of the Dodge Challenger SRT on the streets of New York City.

To do this, he used a 3D model of the vehicle, a New York HDRi Map and Backplate from our collection, and his decades of skill in the field.

This image was produced entirely virtually — that car was never actually on that NYC street!

To see how AI would do, we gave Midjourney 4 the following prompt: “A dodge challenger on the streets of New York City”

We’ll discuss more below, but let’s take a look at an example from another field.

Architectural Visualization with Midjourney--Jay Patel’s Render

Clients often use CGI.Backgrounds’ premium HDRi Maps for architectural visualization, both for visuals and for lighting. 3D designer Jay Patel recently used their HDRi Maps to create a stunning render of an imagined structure.

Based on Patel’s brief, we gave Midjourney the following prompt: “A modern, glass-walled building that is a combination of the Farnsworth House and Bruce Wayne’s house in Batman V Superman, with moody lighting at dusk, beside a rocky shoreline with puddles of water in the foreground”

Here is the result:

Aerospace Visualizations with Midjourney — a Futuristic EVTOL

Designer Doug Didia recently showed how he uses CGI.Backgrounds’ HDRi Maps and Backplates to render futuristic vehicles that don’t yet exist, for his clients in the defense and aerospace industry.

Specifically, he demonstrated how he created a render of an EVTOL aircraft parked beside a warehouse in an urban setting.

We gave a similar task to Midjourney with the prompt: “An unmanned EVTOL parked in the middle of a yellow landing pad circle, in front of an urban warehouse with a large wall of divided-light windows”

Here is what Midjourney produced:

How Are the Results?

Overall, we were impressed with that Midjourney was able to create. From a technical perspective, it’s impressive that AI is able to generate any image from a simple text prompt, much less images that appear to closely resemble their targets.

Still, these images can’t begin to compare with the work of professional 3D designers and artists.

While Midjourney’s Dodge Challenger render captures the basic idea of a Dodge vehicle on a New York street, the specifics are muddled or missing. There are random holes in the hood of the imagined vehicle, the other cars on the road look crushed and unrealistic, and although the vehicle looks Challenger-like, it’s obvious to any car person (or potential car buyer) that this isn’t the real deal.

The shadows are also inaccurate. They seem to how a wooded scene, not a NYC street.

Likewise, the image has a video game-like look that wouldn’t be appropriate in an advertising campaign or product catalog. It’s an impressive creation for a computer, but not even close to the photorealistic rendering Allan created.

Midjourney’s architectural rendering looks a bit more true-to-form, but there are still many issues here. The rendering has a dream-like quality and softness that feels reminiscent of anime or a children’s book illustration. It doesn’t capture the linearity and imposing form and lighting of the structure as imagined by Patel.

Midjourney’s images are also small, at a maximum resolution of 512 pixels. They would need to be upscaled, and even then, they couldn’t achieve the highest resolutions needed for commercial projects. Especially in the arch viz space, a .25 megapixel image wouldn’t cut it.

Finally, although the EVTOL rendering from Midjourney is fanciful and engaging, the system missed a key element of the actual vehicle–the fact that it’s a pilot-less drone. Even with multiple variations of our prompt, we couldn’t get Midjourney to remove the craft’s windows!

Again, this illustrates the challenges with AI imagery. Perhaps an AI image can get you 75% of the way to your final goal. But it’s that last 25% which matters the most.

Whether it’s the specifics of the vehicle model in an automotive rendering, the resolution and light in an arch viz project, or the exact specifications of a defense aircraft, details matter–especially to clients and brands.

There are also major legal questions about the copyright and other implications of AI images. Using these images could result in a brand failing to own the assets created in a campaign or incurring unknown legal liabilities.

For all those reasons, CGI.Backgrounds doesn’t believe that generative AI will threaten the work of their professional 3D designers any time soon. Their ability to create photorealistic renders that exactly match a client’s need–as well as their artistry and decades of experience–set them apart from any AI or automated system.

Uses for Generative AI in 3D Rendering

That said, I could imagine several places in the 3D rendering space where generative AI could come in handy.

For ideation, in particular, tools like Midjourney could provide useful capabilities. Much as a non-technical creative director can use a tool like Adobe’s Substance Stager to create a quick mockup of an idea, a non-technical staff member could use Midjourney to create concepts for a virtual production then hand over to a 3D designer for execution.

Likewise, designers could use a tool like Midjourney to quickly create variations of one of their renders, exploring ideas that might not have occurred to them during the initial ideation process.

For virtual location scouting, designers could use the tools to create a quick concept of a virtual location, and then hand that image over to a location scout, allowing them to find a comparable, licensable asset in a collection like CGI.Backgrounds’.

Finally, generative AI tools could be useful for tasks like storyboarding a virtual production. Designers could combine existing Backplate imagery with 3D models or drawings using generative AI and quickly generate rough storyboards for their productions.

Conclusion

In short, generative AI has its place in the 3D rendering space, but it won’t take over the task of creating final, production-ready renders any time soon, or perhaps ever.

AI image generators are powerful tools, but they provide the most value when placed in the hands of a talented design and production team with the skill (and high-quality HDRi Maps and Backplates) needed to realize complex and mission-critical creative visions.

Want to learn more about my experiments with generative AI for my client CGI.Backgrounds? Sign up for their free newsletter.

Originally published at https://www.cgibackgrounds.com.