avatarBrinda Koushik | Copywriter | Mom | Avid Reader

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

6910

Abstract

ython_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 5.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 2.2 seconds. INFO:main:Done. INFO:main:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:main:Stable Diffusion configured to generate 512x512 images INFO:main:Done. INFO:main:Beginning image generation. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 51/51 [01:50<00:00, 2.17s/it] INFO:main:Generated image has nsfw concept=False</pre></div><p id="77e7">After the program finishes, we could find the file under <code>./output</code></p><figure id="c8ab"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*4Ntawlgr5C5fwo-Ca-2shg.jpeg"><figcaption></figcaption></figure><p id="cd05">However, it takes 3–4 minutes to see result images, rather slow. To generate more variations via adjusting random “seeding” or change the “descriptive text”, all in commands. It could be inconvenient when the text is long。</p><h1 id="a3ea">Step 4. How to make it image generation easier with ML Stable Diffusion?</h1><h2 id="cda1">Let’s build a Web UI</h2><p id="0bb8"><code>gradio</code> is an interesting python library to quickly create simple stunning web UI.</p><figure id="8f88"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*s5nh1On2QyHCr8pWPOL6Wg.jpeg"><figcaption></figcaption></figure><p id="6ab7">Let’s create a <code>web.py</code> with follow content</p><div id="8584"><pre><span class="hljs-keyword">import</span> python_coreml_stable_diffusion.pipeline <span class="hljs-keyword">as</span> pipeline</pre></div><div id="c3c9"><pre><span class="hljs-keyword">import</span> gradio <span class="hljs-keyword">as</span> gr <span class="hljs-title">from</span> diffusers <span class="hljs-keyword">import</span> StableDiffusionPipeline</pre></div><div id="da50"><pre>def init(args): pipeline.logger.<span class="hljs-built_in">info</span>(<span class="hljs-string">"Initializing PyTorch pipe for reference configuration"</span>) pytorch_pipe = StableDiffusionPipeline.from_pretrained(args.model_version, <span class="hljs-attribute">use_auth_token</span>=<span class="hljs-literal">True</span>)</pre></div><div id="3c0d"><pre> user_specified_scheduler = None <span class="hljs-keyword">if</span> <span class="hljs-built_in">args</span>.scheduler <span class="hljs-built_in">is</span> <span class="hljs-keyword">not</span> None: user_specified_scheduler = pipeline.SCHEDULER_MAP[ <span class="hljs-built_in">args</span>.scheduler].from_config(pytorch_pipe.scheduler.config)</pre></div><div id="cb88"><pre> <span class="hljs-attr">coreml_pipe</span> = pipeline.get_coreml_pipe(pytorch_pipe=pytorch_pipe, <span class="hljs-attr">mlpackages_dir</span>=args.i, <span class="hljs-attr">model_version</span>=args.model_version, <span class="hljs-attr">compute_unit</span>=args.compute_unit, <span class="hljs-attr">scheduler_override</span>=user_specified_scheduler) </pre></div><div id="880f"><pre> def infer(<span class="hljs-built_in">prompt</span>, steps): pipeline.logger.info(<span class="hljs-string">"Beginning image generation."</span>) <span class="hljs-built_in">image</span> = coreml_pipe( <span class="hljs-built_in">prompt</span>=<span class="hljs-built_in">prompt</span>, <span class="hljs-built_in">height</span>=coreml_pipe.<span class="hljs-built_in">height</span>, <span class="hljs-built_in">width</span>=coreml_pipe.<span class="hljs-built_in">width</span>, num_inference_steps=steps, ) images = [] images.<span class="hljs-built_in">append</span>(<span class="hljs-built_in">image</span>[<span class="hljs-string">"images"</span>][<span class="hljs-number">0</span>]) <span class="hljs-built_in">return</span> images </pre></div><div id="d83e"><pre> <span class="hljs-attr">demo</span> = gr.Blocks()</pre></div><div id="0310"><pre> with demo: gr.Markdown( <span class="hljs-string">"<center><h1>Core ML Stable Diffusion</h1>Run Stable Diffusion on Apple Silicon with Core ML</center>"</span>) with gr.Group(): with gr.Box(): with gr.Row(): with gr.Column(): with gr.Row(): text = gr.Textbox( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Prompt"</span>, <span class="hljs-attribute">lines</span>=11, <span class="hljs-attribute">placeholder</span>=<span class="hljs-string">"Enter your prompt"</span>, ) with gr.Row(): btn = gr.Button(<span class="hljs-string">"Generate image"</span>) with gr.Row(): steps = gr.Slider(<span class="hljs-attribute">label</span>=<span class="hljs-string">"Steps"</span>, <span class="hljs-attribute">minimum</span>=1, <span class="hljs-attribute">maximum</span>=50, <span class="hljs-attribute">value</span>=10, <span class="hljs-attribute">step</span>=1) with gr.Column(): gallery = gr.Gallery( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Generated image"</span>, <span class="hljs-attribute">elem_id</span>=<span class="hljs-string">"gallery"</span> )</pre></div><div id="6605"><pre> text.submit<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs</span>=gallery)</span> btn.click<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs<

Options

/span>=gallery)</span></pre></div><div id="603f"><pre> demo.launch(<span class="hljs-attribute">debug</span>=<span class="hljs-literal">True</span>, <span class="hljs-attribute">server_name</span>=<span class="hljs-string">"0.0.0.0"</span>) </pre></div><div id="b5d7"><pre><span class="hljs-keyword">if</span> name == <span class="hljs-string">"main"</span>: parser = pipeline<span class="hljs-selector-class">.argparse</span><span class="hljs-selector-class">.ArgumentParser</span>()</pre></div><div id="3ee6"><pre> parser.add_argument( <span class="hljs-string">"-i"</span>, <span class="hljs-attribute">required</span>=<span class="hljs-literal">True</span>, help=(<span class="hljs-string">"Path to input directory with the .mlpackage files generated by "</span> <span class="hljs-string">"python_coreml_stable_diffusion.torch2coreml"</span>)) parser.add_argument( <span class="hljs-string">"--model-version"</span>, <span class="hljs-attribute">default</span>=<span class="hljs-string">"CompVis/stable-diffusion-v1-4"</span>, help= (<span class="hljs-string">"The pre-trained model checkpoint and configuration to restore. "</span> <span class="hljs-string">"For available versions: https://huggingface.co/models?search=stable-diffusion"</span> )) parser.add_argument( <span class="hljs-string">"--compute-unit"</span>, <span class="hljs-attribute">choices</span>=pipeline.get_available_compute_units(), <span class="hljs-attribute">default</span>=<span class="hljs-string">"ALL"</span>, help=(<span class="hljs-string">"The compute units to be used when executing Core ML models. "</span> f<span class="hljs-string">"Options: {pipeline.get_available_compute_units()}"</span>)) parser.add_argument( <span class="hljs-string">"--scheduler"</span>, <span class="hljs-attribute">choices</span>=tuple(pipeline.SCHEDULER_MAP.keys()), <span class="hljs-attribute">default</span>=None, help=(<span class="hljs-string">"The scheduler to use for running the reverse diffusion process. "</span> <span class="hljs-string">"If not specified, the default scheduler from the diffusers pipeline is utilized"</span>))</pre></div><div id="522f"><pre> <span class="hljs-variable">args</span> = <span class="hljs-variable">parser.parse_args</span>() <span class="hljs-function"><span class="hljs-title">init</span>(<span class="hljs-variable">args</span>)</span></pre></div><p id="46e2">Save above <code>web.py</code>into <code>python_coreml_stable_diffusion</code> directory and then run</p><div id="717c"><pre>python -m python_coreml_stable_diffusion.web -i ./models --compute-unit ALL</pre></div><p id="7e7d">Here is the logs after that command</p><div id="1f63"><pre>WARNING:coremltools:Torch version 1.13.0 has not been tested with coremltools. You may run into unexpected errors. Torch 1.12.1 is the most recent version that has been tested. INFO:python_coreml_stable_diffusion.pipeline:Initializing PyTorch pipe <span class="hljs-keyword">for</span> reference configuration Fetching 16 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 16396.01it/s] INFO:python_coreml_stable_diffusion.pipeline:Removed PyTorch pipe to reduce peak memory consumption INFO:python_coreml_stable_diffusion.pipeline:Loading Core ML models <span class="hljs-keyword">in</span> memory from ./models INFO:python_coreml_stable_diffusion.coreml_model:Loading text_encoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_text_encoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 4.4 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading unet mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_unet.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 73.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 6.0 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 1.9 seconds. INFO:python_coreml_stable_diffusion.pipeline:Done. INFO:python_coreml_stable_diffusion.pipeline:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:python_coreml_stable_diffusion.pipeline:Stable Diffusion configured to generate 512x512 images INFO:python_coreml_stable_diffusion.pipeline:Done. Running on <span class="hljs-built_in">local</span> URL: http://0.0.0.0:7860</pre></div><div id="a7f4"><pre><span class="hljs-keyword">To</span> create a <span class="hljs-keyword">public</span> <span class="hljs-keyword">link</span>, <span class="hljs-built_in">set</span> <span class="hljs-string">share=True</span> <span class="hljs-keyword">in</span> <span class="hljs-string">launch()</span>.</pre></div><p id="b1df">We could see that the Web service is starting on port 7860. Let’s open our favourite browser on the address.</p><figure id="db3e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*_HwT7zViq5n86sFosWNt2Q.jpeg"><figcaption></figcaption></figure><p id="4c1a">Let’s test it with “colourful startrails”, then click “Generate image” then wait for the generation to complete and the image will appear on the right side.</p><figure id="fb07"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*wuRf6OR_HPTEpKwshZinHw.jpeg"><figcaption></figcaption></figure><p id="470f">Now, it is much easier. We only need to adjust the text in the “prompt” and a single click would generate image, saving us from command param adjusting or digging file through the directory. In addition, the model is loaded only once unlike previously each invoke will require loading model separately, saving a lot of time too.</p><p id="9f82">If you find the guide helpful, feel free to clap and follow me. Join medium via <a href="https://medium.com/@caodanju/membership">this link</a> to access all premium articles from me and all other awesome writers here on medium.</p></article></body>

Life Lessons

Watch Out! Your Next English Lesson Can Be From Your Kid.

When did my tween get so smart?

Photo by Jonathan Borba on Unsplash

It was a cold and rainy Saturday afternoon post-lunch, and I was trying to escape my children's’ eyes and go write something. As I finished cleaning the kitchen, I was about to sneak into the study and my daughter called me to play with her. She just wanted to read me one of her current favorite books.

She not only loves to read me the books she enjoyed, but she also takes pride in the fact that her mom decided to be a writer. Whenever we meet someone, they tend to ask me, What do I do these days? I sheepishly reply I am just with kids these days, and I get so busy running around them 24/7 as the school is still shut because of covid, and so on… My daughter often nudges me to say that I am a writer, and sometimes if the person we meet is a bit friendly with her, she blabbers it out, taking them by surprise.

Ahem, I start — “It’s less than six months, and I often feel a bit shy to say that I am writing.” It raises eyebrows because I was in corporate for more than a decade, and took a break from work for three years, and now writing? I don't know, but I feel they might judge and think that maybe she hasn’t managed to get a job, so she is doing this. I know I should learn to let go of it, but it is taking some time. My husband warns me, saying, the more clear I am about my career, the better. True.

Coming to the present scenario, she began reading a chapter that we had left off from the book titled “Cat & Mouse in a Haunted House,” a book in the Geronimo Stilton part 1 series by Elisabetta Dami. It is a great story, but my mind was elsewhere. I had to write down some topics that occurred to me and finish editing one post that was sitting for two days. She read a few pages, and I picked my phone lying beside me. She stared at me, and her eyes signaled me to place it back. Lol.

I reluctantly placed it back and tried to follow the storyline because she cross-questions me suddenly. And if I give a wrong answer, she gets upset that I am not listening attentively.

After all, my daughter has been home for more than a month due to a 2nd lockdown in the country we live in, and add to that, she is terribly bored without her friends and school time. It is she who manages my toddler and asks me to write when I have a deadline to finish. And when I have to slip out of the house, for sending a post or dumping the garbage, she is the one who engages the toddler for those minutes, though the husband would be working from home.

I felt guilty that I could not give her enough time and calmed down to give her my full attention.

I listened for the next 3 minutes as my phone beeped about a notification that someone liked my post. I eagerly grabbed my phone, and my daughter snatched it from my hand and placed it on a table away from me. She said, “Mommy if you listen, you can also learn something from this.” I rolled my eyes mentally, then she said, “You can use some phrases in this book in your writings too; it will be helpful. Listen, this one ‘My heart hammered away in my chest’ You can use it somewhere, right?” My ears perked up, and my eyes widened! Then she mentioned some more as she read, “Collapsed in a fit of giggles…”, and then came “my obnoxious cousin…” Oh yes! She was right. It changed my perspective about her instantly! I listened to her more attentively for as long as I could. Speechless that I was…

I often thought my articles should not be full of boring grammatically correct sentences but full of the right emotion and a bit of quirkiness (but I’m sometimes lazy to see the thesaurus or look up synonyms to make it sound better). By not giving my time and attention, I realized I was missing out on the little nuances of life that matter.

There are profound learnings and such examples in simple things around us, but we barely take notice.

It sure broke my ego, melted my heart, and I was a proud mama at that moment. She is a writer’s daughter, after all ;) As I narrated this to my husband, my heart swelled with pride, and he too nodded that she indeed had a point. I know we are never too old to learn but never saw it coming from a 9-year-old.

This post is not to brag about my daughter but to emphasize how we often undermine our child’s capability. It’s surprising how the new generation's thoughts can fuel old minds. Kids are not always attention seekers, but they care deeply about us (or our identity). It’s time we let our children have a mind of their own so that they can have a voice of their own.

Children are often such great teachers, and we often neglect life lessons playing in front of our eyes through them.

Like to binge-read more from the author?

Parenting
Moms
Love
Writers Life
Parenting Advice
Recommended from ReadMedium