Cinematography with Midjourney, Part 3: Camera & Shot Types (AI Image Generation)
Creating Cinematic AI Images with Midjourney
This is the third part of a three-part series on cinematic prompts. In this post we will cover the following:
- Referencing camera types with quality suffixes
- Referencing camera types directly/indirectly
- Referencing basic shot types
- Referencing low/high angle shots & depth of field
- Referencing over-the-shoulder shots
If you missed the first two parts of this series, you can read them here:
Part One: Cinematography with Midjourney, Part 1: Anatomy Of A Cinematic Prompt
Part Two: Cinematography with Midjourney, Part 2: Mis-En-Scène & Lighting (AI Image Generation)
Introduction to Camera Types
Different types of cameras can be used to create different types of look and feel. Footage from analog film cameras, for example, is created through photochemical processes that create a cinematic look due to the “imperfections” of the coloring process (something that digital film cameras, such as the ARRI Alexa, try to mimic; read more about this here). Other types of cameras, such as older Super 8 or 16mm cameras, have an aesthetic based on low resolution and grainy film stock.
Fun fact: you can prompt for some of the early pioneers of cinematography like photographer Eadweard Muybridge, who placed 12 cameras along a horse race track to create a series of sequential photographs in 1878. The man became so famous for “chronophotography of animal locomotion” that you can ask Midjourney to render in the style of his work:
/imagine prompt: Muybridge sequence
When referencing camera types, we have to keep in mind that some camera characteristics might already be included in our prompt even without explicitly mentioning any camera. For example, in style descriptions like this:
film still, bus stop scene, style by 1954 thriller movie — ar 3:2 — seed 3000
The reference to the era already gives Midjourney a baseline to render the image in a way that is consistent with the camera techniques of the time: black and white film. The reference to the genre further indicates elements like composition, low-key lighting, contrast, and so on.
The challenge with explicitly prompting for certain camera types is that you may end up with a picture of a camera in the image and not the look you want to create. There are exceptions when it comes to very popular camera types, such as those by Arnold & Richter (Arri), Paillard-Bolex, or Panavision. However, these are often implied in style descriptions and come with some limitations, as they are mostly tied to a specific type of use or shot for Midjourney.
Traditional and modern camera types
To further illustrate the predefinitions included in references to genre and era, compare the following two prompts and their results:
film still, man on a rooftop, style by 2020s sci fi movie — ar 3:2 — seed 3000
and
film still, man on a rooftop, style by 1950s sci fi movie — ar 3:2 — seed 3000
We see that the respective zeitgeist dictates the rendering style: on the one hand, first-person shooter-inspired shots with modern cameras and color grading; on the other, low-angle shots of a hero with a cliché hairstyle and film grain.
If we were to describe the shot characteristics of the 2020s science fiction film still, we could list the following:
- sharp and detailed
- fine grain
- dramatic colors
The 1950s version could be characterized like this:
- no dramatic colors
- subdued colors
- film grain
Let’s first check what happens when we enforce the 2020s version’s characteristics like so:
film still, man on a rooftop, style by 2020s sci fi movie, sharp and detailed, fine grain, dramatic colors — ar 3:2 — seed 3000
And the 1950s version like this:
film still, man on a rooftop, style by 1950s sci fi movie, film grain, subdued colors — no dramatic lighting — ar 3:2 — seed 3000
Apparently they pretty much stay within the genre/era characteristics.
Now let’s add the 1950s characteristics to the 2020s version and vice versa:
film still, man on a rooftop, style by 2020s sci fi movie, film grain, subdued colors — no dramatic lighting — ar 3:2 — seed 3000
and
film still, man on a rooftop, style by 1950s sci fi movie, sharp and detailed, fine grain, dramatic colors — ar 3:2 — seed 3000
We can see that the genre/era predefinitions are so strong that Midjourney rather turns the results into cartoonish images than allowing 2020 film production characteristics in a 1950s sci-fi movie or adding 1950s technical limitations in a 2020s blockbuster.
But, we can switch styles. We will give the 1950s film still a 2020s look by adding a general camera/quality suffix like “8k, photorealistic”, and on the other hand turn the 2020s into the 1950s by adding a camera/quality suffix like “eastmancolor, photorealistic” (note that there are many suffixes you can use here; the above proofed to be quite robust).
Full prompts:
film still, man on a rooftop, style by 1950s sci fi movie, sharp and detailed, fine grain, dramatic colors, 8k, photorealistic — ar 3:2 — seed 3000
film still, man on a rooftop, style by 2020s sci fi movie, subdued colors, film grain, eastmancolor, photorealistic — no dramatic lighting — ar 3:2 — seed 3000
Technicolor: Then And Now
An interesting note: If you use the phrase “Technicolor, photorealistic” to refer to the popular Technicolor three-strip color process, you won’t get the super-saturated, high-contrast images of the Technicolor era, but rather something that resembles a modern Instagram-filter interpretation of “Technicolor” with lots of cyan and magenta.
film still, man on a rooftop, style by 2020s sci fi movie, subdued colors, film grain, technicolor, photorealistic — no dramatic lighting — ar 3:2 — seed 3000
You can get some good technicolor renders, however, if you make it fit the predefinitions to enforce the historic technicolor reference instead of the cyan/magenta booster. Here are some examples with
film still, [scene description], style by 1940s [genre reference], technicolor, photorealistic — ar 3:2 — seed 3000
Old Film Cameras
There are many camera styles you can experiment with. For those interested in the evolution of colored film and innovations like Pathécolor, Kinemacolor, Technicolor and Eastmancolor, check out this link.
Here’s an intersting one:
film still, bus stop scene, style by 1954 thriller movie, pathécolor — ar 3:2 — seed 3000
Midjourney seemingly “gets” that this has to do with film tinting, but obviously, it’s not really what Pathécolor films look like. We can, however, instruct Midjourney to recreate something very similar to the pathécolor style by describing what we actually want:
film still, bus stop scene, style by 1954 thriller movie, tinted film — ar 3:2 — seed 3000
Of course, this is not the real deal, but it shows us a way to handle the limitations of the current Midjourney model and get good results by describing the characteristics of a particular film/camera type.
Also, keep in mind that a style reference always affects the entire scene. The way diffusion models work, you can’t control individual elements, you always have to create a prompt that generally points to the scene you want to create.
For example, if you want the aesthetic of early color photography and reference the autochrome process (early 1900s), your whole setting will change:
film still, domestic scene, style by autochrome — ar 3:2 — seed 3000
Same with Kodachrome, a popular color film in 1930s–1950s:
film still, domestic scene, style by kodachrome — ar 3:2 — seed 3000
Even referencing a later era doesn’t work properly: set and costumes still resemble the 40s/50s style of kodachrome footage.
film still, domestic scene, style by 2020s thriller, kodachrome — ar 3:2 — seed 3000
Developing Workarounds For Camera Types
Since in most cases, direct reference of camera types does not work yet with Midjourney, we can try to develop workarounds by looking for reference material that is well-known and bears similar characteristics.
For example, to get an “amateur”, “super8” or “dogma” look:
film still, domestic scene, style by disposable camera — ar 3:2 — seed 3000
Or, if you want to have it more “amateur”-like, try “pinhole photography”. If too blurry, this can be balanced out with the quality suffixes mentioned above.
film still, domestic scene, style by pinhole photography — ar 3:2 — seed 3000
Referencing basic shot types
Shot types have such an impact on visual storytelling that a canonical way of using them has developed:
- wide shots to give an overview of the scene,
- medium shots to illustrate action,
- close-ups to focus on a particular character or object,
- extreme close-ups for intense scenes.
Luckily, we can refer to them in a prompt directly like so:
film still, astronaut in the jungle, close up — ar 3:2 — seed 3000
film still, astronaut in the jungle, medium shot — ar 3:2 — seed 3000
film still, astronaut in the jungle, high angle shot, view from above — ar 3:2 — seed 3000
film still, astronaut in the jungle, wide angle — ar 3:2 — seed 3000
Sometimes rendering wide-angle shots will fail because of the description of the scene. In this case, for better results, you should describe what a wide-angle shot would entail besides the character:
film still, astronaut in the jungle, mountain in the background, wide angle — ar 3:2 — seed 3000
or
film still, astronaut in the jungle, plants and large trees in the background, wide angle — ar 3:2 — seed 3000
By the way, one method to counter Midjourney’s tendency to render characters facing away from the viewer is to describe what they are doing, e.g.:
film still, astronaut in the jungle, cooking at campfire, medium shot — ar 3:2 — seed 3000
Low & High Angle Shots
In addition to shot sizes, the camera position is also important to set nuances in the description of characters or scenes:
film still, astronaut in the jungle, low angle shot — ar 3:2 — seed 3000
Since these results are not yet really convincing (apart from the upper right image maybe), we can try to enforce the camera position by using two similar descriptions right next to each other. In this case: “low angle shot” and “viewed from the ground”:
film still, astronaut in the jungle, low-angle shot, viewed from the ground — ar 3:2 — seed 3000
This is much better. Let’s try more enforcement:
film still, astronaut in the jungle, low angle, shot from below, worms eye view — ar 3:2 — seed 3000
Unfortunately, enforcing high-angle shots tends to create aerial views. In this case (as with Dutch angle, POV and some other shot types), re-rolling and selecting the best variations is the only solution at the moment:
film still, astronaut in the jungle, high angle shot — ar 3:2 — seed 3000
film still, astronaut in the jungle, aerial view — ar 3:2 — seed 3000
Depth Of Field
Another important element in visual storytelling is depth of field. Ideally, you want to create a film still that consists of three layers:
- foreground (out of focus),
- middle-ground (where the action takes place),
- background (out of focus or far away in perspective).
This is psychologically the most appealing structure for an image, as it is very close to our perception. Here’s an illustration of this principle in a “wide angle” rendering with the yellow arrows pointing to foreground elements and the blue line marking the background.
In a medium shot without any description of these three layers, the foreground elements are rather sparse. The “film still” prefix forces a basic cinematic look but it could be better:
film still, cowboy in the desert, style by 2020s science fiction, medium shot — ar 3:2 — seed 3000
By mentioning “shallow depth of field”, we reduce the middle-ground and get more toward the three layers we want:
film still, cowboy in the desert, style by 2020s science fiction, medium shot, shallow depth of field — ar 3:2 — seed 3000
Another way to do this is by using typical f-stop values from photography, for example:
film still, cowboy in the desert, style by 2020s science fiction, medium shot, f/2.8 — ar 3:2 — seed 3000
(Note the change in general lighting and color approach, which is more toward the DSLR look.)
Another example, first, without any DOF specification:
film still, street scene, medium shot — ar 3:2 — seed 3000
Then:
film still, street scene, medium shot, shallow depth of field — ar 3:2 — seed 3000
Over-The-Shoulder Shots
Over-The-Shoulder (OTS) shots are not only an integral part of visual storytelling, but they are also great to showcase how enforcing works when prompting Midjourney.
A straightforward approach like this won’t work:
film still, man and woman, over-shoulder shot — ar 3:2 — seed 3000
As before, we now look for other characteristics of an OTS shot we could use to enforce it. An obvious one is the depth of field. But mentioning this is not enough:
film still, man and woman, over-the-shoulder shot, shallow depth of field — ar 3:2 — seed 3000
The prompt is still incoherent. We need the scene to fit the shot type:
film still, man and woman having a conversation, over-the-shoulder shot, shallow depth of field — ar 3:2 — seed 3000
These can be fine-tuned with enforcements and “brackets” (putting similar or the same prompt elements in different places of the prompt)
film still, over-the-shoulder view, man and woman having a conversation, over-the-shoulder shot, shallow depth of field — ar 3:2 — seed 3000
How far did we get?
Remember we wanted to imitate this direct reference to the style of Wes Anderson?
At the end of part one we were able to use this prompt instead:
film still, man in blue jacket sitting at a bar, symmetrical, muted colors, light orange wall — ar 3:2 — seed 3000
After part two, we adjusted the lighting and mise-en-scène to better fit the style of a Wes Anderson film:
film still, man in blue jacket sitting at a bar, symmetrical, muted colors, light orange wall , natural lighting — ar 3:2 — seed 3000 — no shadows
Which brought us pretty close already.
Now with more control over camera & shot types we can further adjust our prompt with these:
- adding “8k, photorealistic”
- adding “wide angle”
Have fun experimenting and thanks for reading! 🙏
If you missed the previous parts, you can check them out here:
Part 1: Anatomy of a cinematic prompt, referencing genre & style
Part 2: Lighting & Mise-en-scène
Part 3: Camera and shot types
Feel free to follow me on Medium or on Twitter for regular posts on AI cinema, screenwriting, and AI art. If you don’t have a Medium account yet, you can test a member account here: https://medium.com/@tristwolff/membership
