avatarChristine Graves

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

6910

Abstract

ython_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 5.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 2.2 seconds. INFO:main:Done. INFO:main:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:main:Stable Diffusion configured to generate 512x512 images INFO:main:Done. INFO:main:Beginning image generation. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 51/51 [01:50<00:00, 2.17s/it] INFO:main:Generated image has nsfw concept=False</pre></div><p id="77e7">After the program finishes, we could find the file under <code>./output</code></p><figure id="c8ab"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*4Ntawlgr5C5fwo-Ca-2shg.jpeg"><figcaption></figcaption></figure><p id="cd05">However, it takes 3–4 minutes to see result images, rather slow. To generate more variations via adjusting random “seeding” or change the “descriptive text”, all in commands. It could be inconvenient when the text is long。</p><h1 id="a3ea">Step 4. How to make it image generation easier with ML Stable Diffusion?</h1><h2 id="cda1">Let’s build a Web UI</h2><p id="0bb8"><code>gradio</code> is an interesting python library to quickly create simple stunning web UI.</p><figure id="8f88"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*s5nh1On2QyHCr8pWPOL6Wg.jpeg"><figcaption></figcaption></figure><p id="6ab7">Let’s create a <code>web.py</code> with follow content</p><div id="8584"><pre><span class="hljs-keyword">import</span> python_coreml_stable_diffusion.pipeline <span class="hljs-keyword">as</span> pipeline</pre></div><div id="c3c9"><pre><span class="hljs-keyword">import</span> gradio <span class="hljs-keyword">as</span> gr <span class="hljs-title">from</span> diffusers <span class="hljs-keyword">import</span> StableDiffusionPipeline</pre></div><div id="da50"><pre>def init(args): pipeline.logger.<span class="hljs-built_in">info</span>(<span class="hljs-string">"Initializing PyTorch pipe for reference configuration"</span>) pytorch_pipe = StableDiffusionPipeline.from_pretrained(args.model_version, <span class="hljs-attribute">use_auth_token</span>=<span class="hljs-literal">True</span>)</pre></div><div id="3c0d"><pre> user_specified_scheduler = None <span class="hljs-keyword">if</span> <span class="hljs-built_in">args</span>.scheduler <span class="hljs-built_in">is</span> <span class="hljs-keyword">not</span> None: user_specified_scheduler = pipeline.SCHEDULER_MAP[ <span class="hljs-built_in">args</span>.scheduler].from_config(pytorch_pipe.scheduler.config)</pre></div><div id="cb88"><pre> <span class="hljs-attr">coreml_pipe</span> = pipeline.get_coreml_pipe(pytorch_pipe=pytorch_pipe, <span class="hljs-attr">mlpackages_dir</span>=args.i, <span class="hljs-attr">model_version</span>=args.model_version, <span class="hljs-attr">compute_unit</span>=args.compute_unit, <span class="hljs-attr">scheduler_override</span>=user_specified_scheduler) </pre></div><div id="880f"><pre> def infer(<span class="hljs-built_in">prompt</span>, steps): pipeline.logger.info(<span class="hljs-string">"Beginning image generation."</span>) <span class="hljs-built_in">image</span> = coreml_pipe( <span class="hljs-built_in">prompt</span>=<span class="hljs-built_in">prompt</span>, <span class="hljs-built_in">height</span>=coreml_pipe.<span class="hljs-built_in">height</span>, <span class="hljs-built_in">width</span>=coreml_pipe.<span class="hljs-built_in">width</span>, num_inference_steps=steps, ) images = [] images.<span class="hljs-built_in">append</span>(<span class="hljs-built_in">image</span>[<span class="hljs-string">"images"</span>][<span class="hljs-number">0</span>]) <span class="hljs-built_in">return</span> images </pre></div><div id="d83e"><pre> <span class="hljs-attr">demo</span> = gr.Blocks()</pre></div><div id="0310"><pre> with demo: gr.Markdown( <span class="hljs-string">"<center><h1>Core ML Stable Diffusion</h1>Run Stable Diffusion on Apple Silicon with Core ML</center>"</span>) with gr.Group(): with gr.Box(): with gr.Row(): with gr.Column(): with gr.Row(): text = gr.Textbox( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Prompt"</span>, <span class="hljs-attribute">lines</span>=11, <span class="hljs-attribute">placeholder</span>=<span class="hljs-string">"Enter your prompt"</span>, ) with gr.Row(): btn = gr.Button(<span class="hljs-string">"Generate image"</span>) with gr.Row(): steps = gr.Slider(<span class="hljs-attribute">label</span>=<span class="hljs-string">"Steps"</span>, <span class="hljs-attribute">minimum</span>=1, <span class="hljs-attribute">maximum</span>=50, <span class="hljs-attribute">value</span>=10, <span class="hljs-attribute">step</span>=1) with gr.Column(): gallery = gr.Gallery( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Generated image"</span>, <span class="hljs-attribute">elem_id</span>=<span class="hljs-string">"gallery"</span> )</pre></div><div id="6605"><pre> text.submit<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs</span>=gallery)</span> btn.click<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs<

Options

/span>=gallery)</span></pre></div><div id="603f"><pre> demo.launch(<span class="hljs-attribute">debug</span>=<span class="hljs-literal">True</span>, <span class="hljs-attribute">server_name</span>=<span class="hljs-string">"0.0.0.0"</span>) </pre></div><div id="b5d7"><pre><span class="hljs-keyword">if</span> name == <span class="hljs-string">"main"</span>: parser = pipeline<span class="hljs-selector-class">.argparse</span><span class="hljs-selector-class">.ArgumentParser</span>()</pre></div><div id="3ee6"><pre> parser.add_argument( <span class="hljs-string">"-i"</span>, <span class="hljs-attribute">required</span>=<span class="hljs-literal">True</span>, help=(<span class="hljs-string">"Path to input directory with the .mlpackage files generated by "</span> <span class="hljs-string">"python_coreml_stable_diffusion.torch2coreml"</span>)) parser.add_argument( <span class="hljs-string">"--model-version"</span>, <span class="hljs-attribute">default</span>=<span class="hljs-string">"CompVis/stable-diffusion-v1-4"</span>, help= (<span class="hljs-string">"The pre-trained model checkpoint and configuration to restore. "</span> <span class="hljs-string">"For available versions: https://huggingface.co/models?search=stable-diffusion"</span> )) parser.add_argument( <span class="hljs-string">"--compute-unit"</span>, <span class="hljs-attribute">choices</span>=pipeline.get_available_compute_units(), <span class="hljs-attribute">default</span>=<span class="hljs-string">"ALL"</span>, help=(<span class="hljs-string">"The compute units to be used when executing Core ML models. "</span> f<span class="hljs-string">"Options: {pipeline.get_available_compute_units()}"</span>)) parser.add_argument( <span class="hljs-string">"--scheduler"</span>, <span class="hljs-attribute">choices</span>=tuple(pipeline.SCHEDULER_MAP.keys()), <span class="hljs-attribute">default</span>=None, help=(<span class="hljs-string">"The scheduler to use for running the reverse diffusion process. "</span> <span class="hljs-string">"If not specified, the default scheduler from the diffusers pipeline is utilized"</span>))</pre></div><div id="522f"><pre> <span class="hljs-variable">args</span> = <span class="hljs-variable">parser.parse_args</span>() <span class="hljs-function"><span class="hljs-title">init</span>(<span class="hljs-variable">args</span>)</span></pre></div><p id="46e2">Save above <code>web.py</code>into <code>python_coreml_stable_diffusion</code> directory and then run</p><div id="717c"><pre>python -m python_coreml_stable_diffusion.web -i ./models --compute-unit ALL</pre></div><p id="7e7d">Here is the logs after that command</p><div id="1f63"><pre>WARNING:coremltools:Torch version 1.13.0 has not been tested with coremltools. You may run into unexpected errors. Torch 1.12.1 is the most recent version that has been tested. INFO:python_coreml_stable_diffusion.pipeline:Initializing PyTorch pipe <span class="hljs-keyword">for</span> reference configuration Fetching 16 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 16396.01it/s] INFO:python_coreml_stable_diffusion.pipeline:Removed PyTorch pipe to reduce peak memory consumption INFO:python_coreml_stable_diffusion.pipeline:Loading Core ML models <span class="hljs-keyword">in</span> memory from ./models INFO:python_coreml_stable_diffusion.coreml_model:Loading text_encoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_text_encoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 4.4 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading unet mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_unet.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 73.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 6.0 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 1.9 seconds. INFO:python_coreml_stable_diffusion.pipeline:Done. INFO:python_coreml_stable_diffusion.pipeline:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:python_coreml_stable_diffusion.pipeline:Stable Diffusion configured to generate 512x512 images INFO:python_coreml_stable_diffusion.pipeline:Done. Running on <span class="hljs-built_in">local</span> URL: http://0.0.0.0:7860</pre></div><div id="a7f4"><pre><span class="hljs-keyword">To</span> create a <span class="hljs-keyword">public</span> <span class="hljs-keyword">link</span>, <span class="hljs-built_in">set</span> <span class="hljs-string">share=True</span> <span class="hljs-keyword">in</span> <span class="hljs-string">launch()</span>.</pre></div><p id="b1df">We could see that the Web service is starting on port 7860. Let’s open our favourite browser on the address.</p><figure id="db3e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*_HwT7zViq5n86sFosWNt2Q.jpeg"><figcaption></figcaption></figure><p id="4c1a">Let’s test it with “colourful startrails”, then click “Generate image” then wait for the generation to complete and the image will appear on the right side.</p><figure id="fb07"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*wuRf6OR_HPTEpKwshZinHw.jpeg"><figcaption></figcaption></figure><p id="470f">Now, it is much easier. We only need to adjust the text in the “prompt” and a single click would generate image, saving us from command param adjusting or digging file through the directory. In addition, the model is loaded only once unlike previously each invoke will require loading model separately, saving a lot of time too.</p><p id="9f82">If you find the guide helpful, feel free to clap and follow me. Join medium via <a href="https://medium.com/@caodanju/membership">this link</a> to access all premium articles from me and all other awesome writers here on medium.</p></article></body>

Using My Writing as an Escape

How creative writing helps me keep my sanity

Photo by Thought Catalog on Unsplash

What’s your idea of the great escape? Is it taking a trip abroad? Is it going camping with the family? Maybe it’s just a quick trip to the store all by yourself. Yes, these are all great ways to escape the mundane world. But my escape is simply in writing.

As a writer, I can be anyone or anything I so desire. I can be the poet who weaves words into visions. I can be an assassin or a goddess or just your average bartender to the dead. It’s whatever I want.

I can also be a teacher to those still figuring this whole writing thing out. Or a student to those who have way more knowledge than I do. I can be the voice of reason in a world of chaos. I can be the shining light at the end of the dark tunnel.

I write a lot of fantasy fiction as it’s my greatest escape. I have the most boring real-life imaginable. I’m a grandmother — married to my high school sweetheart — raising one of our 9 grandchildren — and I take care of people with special needs as my vocation. I have a lot on my plate, but I still need a getaway. 90% of my day is taking care of everyone else around me. I’ve always been that person. But when I’m writing, that part of my life seems to slip away.

Most of the people around me on a regular basis have no idea what I do. They don’t know that I write fiction or poetry. They have no clue that I’m a writer for several publications on this platform, nor have they even heard of this platform. They would be completely shocked if they knew what I did after hours.

Using writing as an escape has given me a chance to get a lot of things out of my head that would normally drive me nuts. I will admit, I don’t publish everything I write. Some of it’s just for me and my sanity, but that’s okay. It clears up room for happier thoughts and more creative ideas.

I’ve also found that I sleep much better after I’ve written my piece. Even if that piece is total crap, it’s out of my head and in front of my face. I can look at it, study it, feel it. Then, I can either polish it up or just put it away. Someday I might need that piece for another idea or maybe it’s just something I needed to get off my chest and wasn’t meant for the rest of the world.

I love the fact that it doesn’t cost me an arm and a leg to escape my reality. I get to live my normal life during the day and become something more at night. I love the fact that I can get lost in words and not have to worry about losing myself in the process. I love the rush I get when I go back and read what I’ve written and it feels as though it were written by someone else.

I love the fact that I can create my own escape.

This is my latest post for the monthly writing challenge as told by Amy Marley. I chose the word “escape” and this is what I came up with. Thanks Amy, this is turning out to be better than I’d hoped. Check out Amy’s latest post in the challenge:

© Christine Graves 2020

Poetry
Creative Writing
Fiction Writing
Writing Challenge
Writing Prompts
Recommended from ReadMedium