avatarAnne Bonfert

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

6910

Abstract

ython_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 5.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 2.2 seconds. INFO:main:Done. INFO:main:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:main:Stable Diffusion configured to generate 512x512 images INFO:main:Done. INFO:main:Beginning image generation. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 51/51 [01:50<00:00, 2.17s/it] INFO:main:Generated image has nsfw concept=False</pre></div><p id="77e7">After the program finishes, we could find the file under <code>./output</code></p><figure id="c8ab"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*4Ntawlgr5C5fwo-Ca-2shg.jpeg"><figcaption></figcaption></figure><p id="cd05">However, it takes 3–4 minutes to see result images, rather slow. To generate more variations via adjusting random “seeding” or change the “descriptive text”, all in commands. It could be inconvenient when the text is long。</p><h1 id="a3ea">Step 4. How to make it image generation easier with ML Stable Diffusion?</h1><h2 id="cda1">Let’s build a Web UI</h2><p id="0bb8"><code>gradio</code> is an interesting python library to quickly create simple stunning web UI.</p><figure id="8f88"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*s5nh1On2QyHCr8pWPOL6Wg.jpeg"><figcaption></figcaption></figure><p id="6ab7">Let’s create a <code>web.py</code> with follow content</p><div id="8584"><pre><span class="hljs-keyword">import</span> python_coreml_stable_diffusion.pipeline <span class="hljs-keyword">as</span> pipeline</pre></div><div id="c3c9"><pre><span class="hljs-keyword">import</span> gradio <span class="hljs-keyword">as</span> gr <span class="hljs-title">from</span> diffusers <span class="hljs-keyword">import</span> StableDiffusionPipeline</pre></div><div id="da50"><pre>def init(args): pipeline.logger.<span class="hljs-built_in">info</span>(<span class="hljs-string">"Initializing PyTorch pipe for reference configuration"</span>) pytorch_pipe = StableDiffusionPipeline.from_pretrained(args.model_version, <span class="hljs-attribute">use_auth_token</span>=<span class="hljs-literal">True</span>)</pre></div><div id="3c0d"><pre> user_specified_scheduler = None <span class="hljs-keyword">if</span> <span class="hljs-built_in">args</span>.scheduler <span class="hljs-built_in">is</span> <span class="hljs-keyword">not</span> None: user_specified_scheduler = pipeline.SCHEDULER_MAP[ <span class="hljs-built_in">args</span>.scheduler].from_config(pytorch_pipe.scheduler.config)</pre></div><div id="cb88"><pre> <span class="hljs-attr">coreml_pipe</span> = pipeline.get_coreml_pipe(pytorch_pipe=pytorch_pipe, <span class="hljs-attr">mlpackages_dir</span>=args.i, <span class="hljs-attr">model_version</span>=args.model_version, <span class="hljs-attr">compute_unit</span>=args.compute_unit, <span class="hljs-attr">scheduler_override</span>=user_specified_scheduler) </pre></div><div id="880f"><pre> def infer(<span class="hljs-built_in">prompt</span>, steps): pipeline.logger.info(<span class="hljs-string">"Beginning image generation."</span>) <span class="hljs-built_in">image</span> = coreml_pipe( <span class="hljs-built_in">prompt</span>=<span class="hljs-built_in">prompt</span>, <span class="hljs-built_in">height</span>=coreml_pipe.<span class="hljs-built_in">height</span>, <span class="hljs-built_in">width</span>=coreml_pipe.<span class="hljs-built_in">width</span>, num_inference_steps=steps, ) images = [] images.<span class="hljs-built_in">append</span>(<span class="hljs-built_in">image</span>[<span class="hljs-string">"images"</span>][<span class="hljs-number">0</span>]) <span class="hljs-built_in">return</span> images </pre></div><div id="d83e"><pre> <span class="hljs-attr">demo</span> = gr.Blocks()</pre></div><div id="0310"><pre> with demo: gr.Markdown( <span class="hljs-string">"<center><h1>Core ML Stable Diffusion</h1>Run Stable Diffusion on Apple Silicon with Core ML</center>"</span>) with gr.Group(): with gr.Box(): with gr.Row(): with gr.Column(): with gr.Row(): text = gr.Textbox( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Prompt"</span>, <span class="hljs-attribute">lines</span>=11, <span class="hljs-attribute">placeholder</span>=<span class="hljs-string">"Enter your prompt"</span>, ) with gr.Row(): btn = gr.Button(<span class="hljs-string">"Generate image"</span>) with gr.Row(): steps = gr.Slider(<span class="hljs-attribute">label</span>=<span class="hljs-string">"Steps"</span>, <span class="hljs-attribute">minimum</span>=1, <span class="hljs-attribute">maximum</span>=50, <span class="hljs-attribute">value</span>=10, <span class="hljs-attribute">step</span>=1) with gr.Column(): gallery = gr.Gallery( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Generated image"</span>, <span class="hljs-attribute">elem_id</span>=<span class="hljs-string">"gallery"</span> )</pre></div><div id="6605"><pre> text.submit<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs</span>=gallery)</span> btn.click<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs<

Options

/span>=gallery)</span></pre></div><div id="603f"><pre> demo.launch(<span class="hljs-attribute">debug</span>=<span class="hljs-literal">True</span>, <span class="hljs-attribute">server_name</span>=<span class="hljs-string">"0.0.0.0"</span>) </pre></div><div id="b5d7"><pre><span class="hljs-keyword">if</span> name == <span class="hljs-string">"main"</span>: parser = pipeline<span class="hljs-selector-class">.argparse</span><span class="hljs-selector-class">.ArgumentParser</span>()</pre></div><div id="3ee6"><pre> parser.add_argument( <span class="hljs-string">"-i"</span>, <span class="hljs-attribute">required</span>=<span class="hljs-literal">True</span>, help=(<span class="hljs-string">"Path to input directory with the .mlpackage files generated by "</span> <span class="hljs-string">"python_coreml_stable_diffusion.torch2coreml"</span>)) parser.add_argument( <span class="hljs-string">"--model-version"</span>, <span class="hljs-attribute">default</span>=<span class="hljs-string">"CompVis/stable-diffusion-v1-4"</span>, help= (<span class="hljs-string">"The pre-trained model checkpoint and configuration to restore. "</span> <span class="hljs-string">"For available versions: https://huggingface.co/models?search=stable-diffusion"</span> )) parser.add_argument( <span class="hljs-string">"--compute-unit"</span>, <span class="hljs-attribute">choices</span>=pipeline.get_available_compute_units(), <span class="hljs-attribute">default</span>=<span class="hljs-string">"ALL"</span>, help=(<span class="hljs-string">"The compute units to be used when executing Core ML models. "</span> f<span class="hljs-string">"Options: {pipeline.get_available_compute_units()}"</span>)) parser.add_argument( <span class="hljs-string">"--scheduler"</span>, <span class="hljs-attribute">choices</span>=tuple(pipeline.SCHEDULER_MAP.keys()), <span class="hljs-attribute">default</span>=None, help=(<span class="hljs-string">"The scheduler to use for running the reverse diffusion process. "</span> <span class="hljs-string">"If not specified, the default scheduler from the diffusers pipeline is utilized"</span>))</pre></div><div id="522f"><pre> <span class="hljs-variable">args</span> = <span class="hljs-variable">parser.parse_args</span>() <span class="hljs-function"><span class="hljs-title">init</span>(<span class="hljs-variable">args</span>)</span></pre></div><p id="46e2">Save above <code>web.py</code>into <code>python_coreml_stable_diffusion</code> directory and then run</p><div id="717c"><pre>python -m python_coreml_stable_diffusion.web -i ./models --compute-unit ALL</pre></div><p id="7e7d">Here is the logs after that command</p><div id="1f63"><pre>WARNING:coremltools:Torch version 1.13.0 has not been tested with coremltools. You may run into unexpected errors. Torch 1.12.1 is the most recent version that has been tested. INFO:python_coreml_stable_diffusion.pipeline:Initializing PyTorch pipe <span class="hljs-keyword">for</span> reference configuration Fetching 16 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 16396.01it/s] INFO:python_coreml_stable_diffusion.pipeline:Removed PyTorch pipe to reduce peak memory consumption INFO:python_coreml_stable_diffusion.pipeline:Loading Core ML models <span class="hljs-keyword">in</span> memory from ./models INFO:python_coreml_stable_diffusion.coreml_model:Loading text_encoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_text_encoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 4.4 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading unet mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_unet.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 73.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 6.0 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 1.9 seconds. INFO:python_coreml_stable_diffusion.pipeline:Done. INFO:python_coreml_stable_diffusion.pipeline:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:python_coreml_stable_diffusion.pipeline:Stable Diffusion configured to generate 512x512 images INFO:python_coreml_stable_diffusion.pipeline:Done. Running on <span class="hljs-built_in">local</span> URL: http://0.0.0.0:7860</pre></div><div id="a7f4"><pre><span class="hljs-keyword">To</span> create a <span class="hljs-keyword">public</span> <span class="hljs-keyword">link</span>, <span class="hljs-built_in">set</span> <span class="hljs-string">share=True</span> <span class="hljs-keyword">in</span> <span class="hljs-string">launch()</span>.</pre></div><p id="b1df">We could see that the Web service is starting on port 7860. Let’s open our favourite browser on the address.</p><figure id="db3e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*_HwT7zViq5n86sFosWNt2Q.jpeg"><figcaption></figcaption></figure><p id="4c1a">Let’s test it with “colourful startrails”, then click “Generate image” then wait for the generation to complete and the image will appear on the right side.</p><figure id="fb07"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*wuRf6OR_HPTEpKwshZinHw.jpeg"><figcaption></figcaption></figure><p id="470f">Now, it is much easier. We only need to adjust the text in the “prompt” and a single click would generate image, saving us from command param adjusting or digging file through the directory. In addition, the model is loaded only once unlike previously each invoke will require loading model separately, saving a lot of time too.</p><p id="9f82">If you find the guide helpful, feel free to clap and follow me. Join medium via <a href="https://medium.com/@caodanju/membership">this link</a> to access all premium articles from me and all other awesome writers here on medium.</p></article></body>

WRITING PROMPT RESPONSE

My Road to Independence

Walking from one home to another at the age of four

This is me. The picture was taken by my mom or dad. | Credit: Anne Bonfert

Independence. Such a big word is used for many things in life. The independence of countries, cities, or people. Children are dependent on their parents. Up to a certain age. Or maybe forever. Some become independent at an early age. Some never really manage not to depend on others.

When I read the prompt on our early experience of becoming an independent people, I had to think back at my childhood. That one specific day.

I was just there to transmit a message. This was before cellphones existed and before a landline was installed in our new home. I was the only one able to bring a message from my mom in the old apartment to my dad, in the new flat.

I was four years old.

We were in the move between the apartment my parents were renting and the new flat they had bought. My dad was working in the new flat fixing up the bathroom together with friends of his.

And I was the one who had to deliver the message.

It might not have been a very far walk. For you. But for a four year old, it was more than a long-distance run. I had to turn off 3 times. It wasn’t just walking down the same road.

My sister wasn’t even a year old and my mom stayed behind in the flat with her. I remember she gave me a shopping bag with a letter for my dad. The letter said something as basic as “your uncle is coming to help”. I think, if I remember correctly.

The bag was almost as big as I was. And I want to claim she also put a bottle of beer in the bag. For my dad. She told me not to talk to anyone. Just walk straight to the new flat.

And off I went.

I knew the way. There was no doubt about that. My mom was sure I’d find it. And I was convinced too. I remember walking down the road. Very vaguely. But I do remember I felt proud.

I wasn’t scared. Or worried. I was just proud my mom had sent me.

I also remember handing that bag to my dad who was surely more worried seeing I had walked down on my own. But it was always like that. My mom pushed me early into my independence. My dad was rather cautious.

Together, they taught me the perfect balance between courage and caution.

It’s why I got independent at an early age while not doing too many silly mistakes.

“Independence isn’t doing your own thing; it is doing the right thing on your own.” — Kim John Payne

I often take a bit longer to respond to writing prompts but here is my response to Ellie Jacobson’s call for our independence stories. Thank you for the inspiration.

More random writing prompt responses:

Join my email list here if you would like to read more photo essays or sign up for the Medium membership to receive unlimited access to my and other writers’ stories out here (I will receive a commission fee in return).

Independence
Writing Prompts
Inspiration
This Happened To Me
Memories
Recommended from ReadMedium
avatarSufyan Maan, M.Eng
What Happens When You Start Reading Every Day

Think before you speak. Read before you think. — Fran Lebowitz

7 min read
avatarKatharine Esty PhD
What’s Too Old?

My outlook on the future as I turn 90

6 min read
avatarKarolina Kozmana
Common side effects of not drinking

By rejecting alcohol, you reject something very human, an extra limb that we have collectively grown to deal with reality and with each…

10 min read
avatarNinad Kulkarni
First Backpacking Trip To Rajasthan

Travelogue from a time when it wasn’t as conventional as it is today.

9 min read
avatarnabil
What does it feel to be loved?

I just want to be loved the way i love.

6 min read
avatarDevon Price
Laziness Does Not Exist

Psychological research is clear: when people procrastinate, there's usually a good reason

11 min read