How to come up with good prompts for AI image generation
In this post, I’ll teach you how to create good prompts for generating AI art work images. I will base my method on Stable Diffusion, an open-source text-to-image AI model.
(If you are new to AI art, check out my Quick Start Guide and Stable Diffusion prompt builder.)
What is Stable Diffusion?
Stable Diffusion is a text-to-image AI model. It is trained on 2.3 billion image and text description pairs. Because it has seen so much, the model encodes relationship between image pixel values and it’s text descriptions.
As a result, if you put in description like “A Photo of a cat sitting on top of a building”, it would give you images like these:
You may be thinking what’s so special about these images? Couldn’t we get millions of them in a Google search? What’s intriguing about this technology is that you can prompt the model to generate high quality images that do not exist before. For example, you can ask for a portrait painting of Emma Watson by the 19th century American painter John Singer Sargent:
It is incredible that such images can be produced from keyword-pixel correlations! What’s mind-boggling is that it gets the artistic style, faces (which our brains are very unforgiving of tiny mistakes) and shadows correctly, and blends them all together in an aesthetically pleasing manner. I believe that the wonder of large numbers is beyond the comprehension of human minds.
What so special about Stable Diffusion?
This year we have seen a few image generation AI such as DALLE 2 and MidJourney. They too are capable of generating stunning images from text prompts. What’s so special about Stable Diffusion are
- Open-source.
- Low computer hardware requirement.
The implication of these two together is big. You can download the model and run it on your local computer. Now there are PC and Apple version available. There’s an explosion of free-to-use image generation AI powered by Stable Diffusion. The low cost of run allows entrepreneurs to explore Freemium or Ad-supported business models. The end result is going to be making the AI technology more accessible.
Where can I try my prompts?
The easiest way to use Stable Diffusion is via DreamStudio.AI. It is from the creator of Stable Diffusion. You will get some free credits after signing up.
Anatomy of a good prompt
There are proven techniques to generate high quality, specific images. Your prompt should cover most if not all of these areas
- Subject (required)
- Medium
- Style
- Artist
- Website
- Resolution
- Additional details
- Color
First you will need a description of the subject with as much detail as possible. E.g.
Subject
A young woman with light blue dress sitting next to a wooden window reading a book.
We got the following image, which matches the prompt pretty well.
We can be more specific. Let’s add a medium. Some examples are: digital painting, photograph, oil painting. Let’s use
Medium
Digital painting
The new prompt is
Digital painting of a young woman with light blue dress sitting next to a wooden window reading a book
The resulting image is
You can see the image changes from photograph to digital art.
You get the idea. Let’s define the rest of them
Artist
by Stanley Artgerm Lau
Website
artstation
Resolution
8k
Additional details
extremely detailed, ornate, cinematic lighting
color
vivid
Putting them all together, the prompt is
Digital painting of a young woman with light blue dress sitting next to a wooden window reading a book, by Stanley Artgerm Lau, artstation, 8k, extremely detailed, ornate, cinematic lighting, vivid.
which generates this image:
By adding keywords to the prompt, we can engineer the image to get the style we want.
Tips for good prompts
- Be specific in subject.
- Use multiple exclamation marks !! to stress a word and brackets () to reduce its strength.
- Use appropriate medium type consistent with the artist.
- Artist name is a very strong style modifier. Use wisely.
- Experiment with blending styles.
- Use websites like Lexica to study other people’s prompts. If you like a particular image, use the prompt as starting point.
Some good keywords for you
Below are some of my favorite keywords and their effects. (Tested with Stable Diffusion v1.4)
Enjoy!
Medium
Medium defines a category of the artwork.
Style
These keywords further refine the art style.
Artist
Mentioning the artist in the prompt is a strong effect. Study their work and choose wisely.
Website
Mentioning an art or photo site is a strong effect, probably because each site has its niche genre.
Resolution
Additional details
Add specific details to your image.
Color
Add additional color scheme to the image.
Summary
In this post, we have gone through the basic structure of a good prompt. This should be used as a guide rather than rules. The Stable Diffusion model is very flexible. Let it surprise you with some creative combination of keywords!
If you have problem generating stunning artworks, this Stable Diffusion prompt generator would be able to help you. In the next post, I will show you how to make this prompt generator using Notion.
You may also like:
Follow me on Twitter to get content like this.
Consider using my products to support me
- Stable Diffusion Quick Start Guide — Generate AI images in a minute
- Prompt builder for Stable Diffusion: AI Image made easy
- Shuffled gallery for AI artwork
- Markdown Keyboard for iPhone and iPad
- Notion Keyboard — Notionkey for iPhone and iPad
(This article first appears in my blog)