Free AI web copilot to create summaries, insights and extended knowledge, download it at here

Abstract

c="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*4XRu2Cy3CAXcoHepVoJUNQ.png"><figcaption></figcaption></figure><figure id="3f6d"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AzJT2M7U0Swxt9qS-05aEg.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>Or, by creating & changing an image’s segmentation layer, change ear positions or framing:<figure id="8cc5"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*VhIdgRWX0bSivfgyYYaYoQ.png"><figcaption></figcaption></figure><figure id="93b1"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*a3xkl1wHTRHSYryypjJ-sg.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>Want to rotate a single element in any given image? No problem, just create a segmentation layer that indicates image translation:<figure id="1559"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*sp8JRFo3Bg_KXDkZjidYmw.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>What about color palettes? Couldn’t we just extract or create those and then re-compose images with altered palettes? You are damn right:<figure id="4548"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*kZv8OmQ5VF3oJdbHZ37r3A.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>What about the good old text descriptions? Well, changing the text description layer of an image works similarly to traditional AI image generation tools (img2img), but this time the preservation of the image quality and composition is next-level. I mean, look at these:<figure id="fe5c"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*V3XNsto7UUgEIQK0B1KLzw.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure><h2 id="09dc">Combination with masking</h2>If you thought playing around with one representational layer is mind-blowing, then look at what happens when they get combined with masking:<figure id="b5ae"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*z6R2z4F8yFagDrzpCZHhDw.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.

Options

09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>In the above examples, the masked area gets affected without changing the quality of the image. The first row changes the “text description” layer, the second row alters the “color palette” layer. The results speak for themselves, I guess.<h2 id="138a">Combining layers</h2>Things get unbelievably amazing when it gets to combining multiple representational layers. For example, combining text & palette:<figure id="7fc5"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*PgN6MhrSKH5CqVNE1Jqvjg.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>Combining depthmap or sketch and text:<figure id="f797"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*F5MJm5QX6hfdrCiohYiC4g.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>Combining depthmap, sketch, image and palette:<figure id="ea90"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*X0dMytsxzu3NPVck5DdNmQ.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>Combining segmentation, sketch, image and palette:<figure id="f800"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*oxR2-c-XAbYS0UmAWKaKUA.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>Or, and with this I leave you in awe and anticipation of the release of this new model, the combination of completely different and formerly contradictory image, palette, depth map, sketch and segement layers:<figure id="bfda"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*kg_mYWIWRLL9mbv9yOtaaQ.png"><figcaption>Link to the original paper: <a href="https://arxiv.org/abs/2302.09778">https://arxiv.org/abs/2302.09778</a></figcaption></figure>If you want to stay up-to-date, follow me on <a href="http://twitter.com/tristwolff">Twitter</a> or Medium or <a href="https://medium.com/@tristwolff/membership">use my referral link to get full access to all my articles</a> (and those of thousands of other writers).And if you like my content, why not leave a “clap” at the end of this article, so more people can see it?</article></body>

Introducing “Composer”: The Latest Breakthrough In AI Image Generation

Are We Ready For This?

A new paradigm for AI image generation offers far more flexibility and control than anything before. Without sacrificing the quality of AI-generated images, a new framework called “Composer” allows the controlling and manipulation of image elements on a completely new level. Say hello to your new AI paradigm: “Compositionality”!

What’s Compositionality?

The idea behind this groundbreaking new approach is this:

break down an image into some representative layers, such as text description, depthmaps, style, semantics, color palettes, etc.,
then recombine these layers or change them to generate a new version of the image

“Wait a minute, doesn’t that mean that there are countless ways to combine these layers? “

Exactly!

And that’s what creates a vast and unprecedented design space for custom content creation!

In their paper, the authors show how they worked with eight representational layers, but theoretically, there could be many more. However, with the eight representation layers alone, “Composer” is already mind-blowing! Let’s have a look.

Changing single representational layers

Let’s first look at the following representation layers:

sketch
segmentation
color palette
text description

By creating or extracting an image’s sketch layer, altering it, and then using it to re-compose the image, you can do such amazing things as adding clouds or changing leg positions:

Link to the original paper: https://arxiv.org/abs/2302.09778

Or, by creating & changing an image’s segmentation layer, change ear positions or framing:

Want to rotate a single element in any given image? No problem, just create a segmentation layer that indicates image translation:

What about color palettes? Couldn’t we just extract or create those and then re-compose images with altered palettes? You are damn right:

What about the good old text descriptions? Well, changing the text description layer of an image works similarly to traditional AI image generation tools (img2img), but this time the preservation of the image quality and composition is next-level. I mean, look at these:

Combination with masking

If you thought playing around with one representational layer is mind-blowing, then look at what happens when they get combined with masking:

In the above examples, the masked area gets affected without changing the quality of the image. The first row changes the “text description” layer, the second row alters the “color palette” layer. The results speak for themselves, I guess.

Combining layers

Things get unbelievably amazing when it gets to combining multiple representational layers. For example, combining text & palette:

Combining depthmap or sketch and text:

Combining depthmap, sketch, image and palette:

Combining segmentation, sketch, image and palette:

Or, and with this I leave you in awe and anticipation of the release of this new model, the combination of completely different and formerly contradictory image, palette, depth map, sketch and segement layers:

If you want to stay up-to-date, follow me on Twitter or Medium or use my referral link to get full access to all my articles (and those of thousands of other writers).

And if you like my content, why not leave a “clap” at the end of this article, so more people can see it?