avatarBill Myers

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

6910

Abstract

ython_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 5.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 2.2 seconds. INFO:main:Done. INFO:main:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:main:Stable Diffusion configured to generate 512x512 images INFO:main:Done. INFO:main:Beginning image generation. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 51/51 [01:50<00:00, 2.17s/it] INFO:main:Generated image has nsfw concept=False</pre></div><p id="77e7">After the program finishes, we could find the file under <code>./output</code></p><figure id="c8ab"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*4Ntawlgr5C5fwo-Ca-2shg.jpeg"><figcaption></figcaption></figure><p id="cd05">However, it takes 3–4 minutes to see result images, rather slow. To generate more variations via adjusting random “seeding” or change the “descriptive text”, all in commands. It could be inconvenient when the text is long。</p><h1 id="a3ea">Step 4. How to make it image generation easier with ML Stable Diffusion?</h1><h2 id="cda1">Let’s build a Web UI</h2><p id="0bb8"><code>gradio</code> is an interesting python library to quickly create simple stunning web UI.</p><figure id="8f88"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*s5nh1On2QyHCr8pWPOL6Wg.jpeg"><figcaption></figcaption></figure><p id="6ab7">Let’s create a <code>web.py</code> with follow content</p><div id="8584"><pre><span class="hljs-keyword">import</span> python_coreml_stable_diffusion.pipeline <span class="hljs-keyword">as</span> pipeline</pre></div><div id="c3c9"><pre><span class="hljs-keyword">import</span> gradio <span class="hljs-keyword">as</span> gr <span class="hljs-title">from</span> diffusers <span class="hljs-keyword">import</span> StableDiffusionPipeline</pre></div><div id="da50"><pre>def init(args): pipeline.logger.<span class="hljs-built_in">info</span>(<span class="hljs-string">"Initializing PyTorch pipe for reference configuration"</span>) pytorch_pipe = StableDiffusionPipeline.from_pretrained(args.model_version, <span class="hljs-attribute">use_auth_token</span>=<span class="hljs-literal">True</span>)</pre></div><div id="3c0d"><pre> user_specified_scheduler = None <span class="hljs-keyword">if</span> <span class="hljs-built_in">args</span>.scheduler <span class="hljs-built_in">is</span> <span class="hljs-keyword">not</span> None: user_specified_scheduler = pipeline.SCHEDULER_MAP[ <span class="hljs-built_in">args</span>.scheduler].from_config(pytorch_pipe.scheduler.config)</pre></div><div id="cb88"><pre> <span class="hljs-attr">coreml_pipe</span> = pipeline.get_coreml_pipe(pytorch_pipe=pytorch_pipe, <span class="hljs-attr">mlpackages_dir</span>=args.i, <span class="hljs-attr">model_version</span>=args.model_version, <span class="hljs-attr">compute_unit</span>=args.compute_unit, <span class="hljs-attr">scheduler_override</span>=user_specified_scheduler) </pre></div><div id="880f"><pre> def infer(<span class="hljs-built_in">prompt</span>, steps): pipeline.logger.info(<span class="hljs-string">"Beginning image generation."</span>) <span class="hljs-built_in">image</span> = coreml_pipe( <span class="hljs-built_in">prompt</span>=<span class="hljs-built_in">prompt</span>, <span class="hljs-built_in">height</span>=coreml_pipe.<span class="hljs-built_in">height</span>, <span class="hljs-built_in">width</span>=coreml_pipe.<span class="hljs-built_in">width</span>, num_inference_steps=steps, ) images = [] images.<span class="hljs-built_in">append</span>(<span class="hljs-built_in">image</span>[<span class="hljs-string">"images"</span>][<span class="hljs-number">0</span>]) <span class="hljs-built_in">return</span> images </pre></div><div id="d83e"><pre> <span class="hljs-attr">demo</span> = gr.Blocks()</pre></div><div id="0310"><pre> with demo: gr.Markdown( <span class="hljs-string">"<center><h1>Core ML Stable Diffusion</h1>Run Stable Diffusion on Apple Silicon with Core ML</center>"</span>) with gr.Group(): with gr.Box(): with gr.Row(): with gr.Column(): with gr.Row(): text = gr.Textbox( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Prompt"</span>, <span class="hljs-attribute">lines</span>=11, <span class="hljs-attribute">placeholder</span>=<span class="hljs-string">"Enter your prompt"</span>, ) with gr.Row(): btn = gr.Button(<span class="hljs-string">"Generate image"</span>) with gr.Row(): steps = gr.Slider(<span class="hljs-attribute">label</span>=<span class="hljs-string">"Steps"</span>, <span class="hljs-attribute">minimum</span>=1, <span class="hljs-attribute">maximum</span>=50, <span class="hljs-attribute">value</span>=10, <span class="hljs-attribute">step</span>=1) with gr.Column(): gallery = gr.Gallery( <span class="hljs-attribute">label</span>=<span class="hljs-string">"Generated image"</span>, <span class="hljs-attribute">elem_id</span>=<span class="hljs-string">"gallery"</span> )</pre></div><div id="6605"><pre> text.submit<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs</span>=gallery)</span> btn.click<span class="hljs-params">(infer, <span class="hljs-attr">inputs</span>=[text, steps], <span class="hljs-attr">outputs<

Options

/span>=gallery)</span></pre></div><div id="603f"><pre> demo.launch(<span class="hljs-attribute">debug</span>=<span class="hljs-literal">True</span>, <span class="hljs-attribute">server_name</span>=<span class="hljs-string">"0.0.0.0"</span>) </pre></div><div id="b5d7"><pre><span class="hljs-keyword">if</span> name == <span class="hljs-string">"main"</span>: parser = pipeline<span class="hljs-selector-class">.argparse</span><span class="hljs-selector-class">.ArgumentParser</span>()</pre></div><div id="3ee6"><pre> parser.add_argument( <span class="hljs-string">"-i"</span>, <span class="hljs-attribute">required</span>=<span class="hljs-literal">True</span>, help=(<span class="hljs-string">"Path to input directory with the .mlpackage files generated by "</span> <span class="hljs-string">"python_coreml_stable_diffusion.torch2coreml"</span>)) parser.add_argument( <span class="hljs-string">"--model-version"</span>, <span class="hljs-attribute">default</span>=<span class="hljs-string">"CompVis/stable-diffusion-v1-4"</span>, help= (<span class="hljs-string">"The pre-trained model checkpoint and configuration to restore. "</span> <span class="hljs-string">"For available versions: https://huggingface.co/models?search=stable-diffusion"</span> )) parser.add_argument( <span class="hljs-string">"--compute-unit"</span>, <span class="hljs-attribute">choices</span>=pipeline.get_available_compute_units(), <span class="hljs-attribute">default</span>=<span class="hljs-string">"ALL"</span>, help=(<span class="hljs-string">"The compute units to be used when executing Core ML models. "</span> f<span class="hljs-string">"Options: {pipeline.get_available_compute_units()}"</span>)) parser.add_argument( <span class="hljs-string">"--scheduler"</span>, <span class="hljs-attribute">choices</span>=tuple(pipeline.SCHEDULER_MAP.keys()), <span class="hljs-attribute">default</span>=None, help=(<span class="hljs-string">"The scheduler to use for running the reverse diffusion process. "</span> <span class="hljs-string">"If not specified, the default scheduler from the diffusers pipeline is utilized"</span>))</pre></div><div id="522f"><pre> <span class="hljs-variable">args</span> = <span class="hljs-variable">parser.parse_args</span>() <span class="hljs-function"><span class="hljs-title">init</span>(<span class="hljs-variable">args</span>)</span></pre></div><p id="46e2">Save above <code>web.py</code>into <code>python_coreml_stable_diffusion</code> directory and then run</p><div id="717c"><pre>python -m python_coreml_stable_diffusion.web -i ./models --compute-unit ALL</pre></div><p id="7e7d">Here is the logs after that command</p><div id="1f63"><pre>WARNING:coremltools:Torch version 1.13.0 has not been tested with coremltools. You may run into unexpected errors. Torch 1.12.1 is the most recent version that has been tested. INFO:python_coreml_stable_diffusion.pipeline:Initializing PyTorch pipe <span class="hljs-keyword">for</span> reference configuration Fetching 16 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 16396.01it/s] INFO:python_coreml_stable_diffusion.pipeline:Removed PyTorch pipe to reduce peak memory consumption INFO:python_coreml_stable_diffusion.pipeline:Loading Core ML models <span class="hljs-keyword">in</span> memory from ./models INFO:python_coreml_stable_diffusion.coreml_model:Loading text_encoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_text_encoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 4.4 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading unet mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_unet.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 73.5 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading a CoreML model through coremltools triggers compilation every time. The Swift package we provide uses precompiled Core ML models (.mlmodelc) to avoid compile-on-load. INFO:python_coreml_stable_diffusion.coreml_model:Loading vae_decoder mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_vae_decoder.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 6.0 seconds. INFO:python_coreml_stable_diffusion.coreml_model:Loading safety_checker mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Loading ./models/Stable_Diffusion_version_CompVis_stable-diffusion-v1-4_safety_checker.mlpackage INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 1.9 seconds. INFO:python_coreml_stable_diffusion.pipeline:Done. INFO:python_coreml_stable_diffusion.pipeline:Initializing Core ML pipe <span class="hljs-keyword">for</span> image generation INFO:python_coreml_stable_diffusion.pipeline:Stable Diffusion configured to generate 512x512 images INFO:python_coreml_stable_diffusion.pipeline:Done. Running on <span class="hljs-built_in">local</span> URL: http://0.0.0.0:7860</pre></div><div id="a7f4"><pre><span class="hljs-keyword">To</span> create a <span class="hljs-keyword">public</span> <span class="hljs-keyword">link</span>, <span class="hljs-built_in">set</span> <span class="hljs-string">share=True</span> <span class="hljs-keyword">in</span> <span class="hljs-string">launch()</span>.</pre></div><p id="b1df">We could see that the Web service is starting on port 7860. Let’s open our favourite browser on the address.</p><figure id="db3e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*_HwT7zViq5n86sFosWNt2Q.jpeg"><figcaption></figcaption></figure><p id="4c1a">Let’s test it with “colourful startrails”, then click “Generate image” then wait for the generation to complete and the image will appear on the right side.</p><figure id="fb07"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*wuRf6OR_HPTEpKwshZinHw.jpeg"><figcaption></figcaption></figure><p id="470f">Now, it is much easier. We only need to adjust the text in the “prompt” and a single click would generate image, saving us from command param adjusting or digging file through the directory. In addition, the model is loaded only once unlike previously each invoke will require loading model separately, saving a lot of time too.</p><p id="9f82">If you find the guide helpful, feel free to clap and follow me. Join medium via <a href="https://medium.com/@caodanju/membership">this link</a> to access all premium articles from me and all other awesome writers here on medium.</p></article></body>

Medium

How to Describe Medium to Everybody

You can read or publish high-quality articles at any time. Authors include professional writers and journalists from around the world.

Photo by NeONBRAND on Unsplash

I always said, “I explain things so clearly that a vice president can understand.” For some reason, most people don’t agree with me and some even laugh. My brother said, “Oh, no! You asked him a question!”

Explaining Medium.com to friends and some members of my critique group, the Wannabee Writers Club in The Villages, Florida, has been more difficult than expected. Almost all are published book authors and the styles are different.

For those not familiar with Medium.com:

According to its help page, “Medium is a publishing platform where people can read important, insightful stories on the topics that matter most to them and share ideas with the world.”

There’s a bit more to the site than that. Non-subscribers are limited to about five reads per month. Subscribers, at five dollars per month, have unlimited reads and can submit articles — as many as they want.

Authors who include their articles in the Partner Program get paid, based on popularity. Some make enough to support themselves, but it is not easy. You have to be either very popular, get curated often, or publish daily.

Starting October 28, 2019, popularity will be based on read time instead of claps.

Medium curators decide which stories will be featured, but all stories are published unless they violate Medium guidelines, like advertising or advocating violence. There is no advertising on Medium.

Having a story curated is like a reporter having their story put on the first page of a section in the newspaper. Currently, I counted twenty sections, or Medium Topics. Non-curated stories still appear in the network list of your followers.

My experience

For me, it’s been a dream come true. Unlike a blog or social media, the articles are professionally written, on nearly any subject. One section is devoted to improving your writing. In some ways, it feels to me like a humongous newspaper or high-class magazine, without a newspaper’s limitations.

Unlike reporters, you are encouraged to include personal experiences and viewpoints. Readers can comment, too. Most readers, unlike politicians answering a question, do an excellent job of sticking to the story with their comments.

Readers follow writers with interesting articles. I follow authors who write about subjects unfamiliar to me or from a different point of view. I like authors that cause me to think differently about something, even if I don’t agree with them. I even follow some poets.

For example, I enjoyed the story, “My Mother Cooked for Her Husband, Now She Cooks for Herself” by Ucheoma Onwutuebe, which demonstrated how different the culture is for women in Nigeria.

We follow each other. I have followers from all over the world, with various education levels, including PhDs, many different professions, and wide areas of expertise, according to their profiles.

Writing and Publishing Process

For me, the writing process changed significantly using the Medium editor vs. MS Word. The editor has the same spell-check. It always saves frequently, so I don’t have to think about it or change settings. Finally, it will read back the story aloud upon request. It even let me change from the gruff male voice to a female voice. (The read-back feature stopped after the latest upgrade. 8/25/2020).

Many other free apps help polish the story for publication.

Headline Analyzer analyzes my headline to determine the Emotional Marketing Value, as a percentage. Grammarly checks the text for spelling, punctuation, vague phrases, better wording, and grammatical mistakes. I run everybody’s submissions for the writers club through Grammarly, if I have time before the meeting.

Medium highly recommends a heading picture, with a citation, at the top of an article. Some publications require one. I usually get free pictures from Unsplash. The Medium help section provides other free sources like Pexels, Pixabay, and the Gender Spectrum Collection.

Unless there is a reason to publish immediately, I read the article to the Writers Club for critiques. After making suggested changes, I hit the “Publish” button.

The article is published instantly and available to my followers unless I submitted it as a draft to a publication. I can also send a ‘friend’ link to non-members on social media.

The curators sometimes take a day or more to decide if the article will be featured. Poor writing quality or articles about Medium will not be featured.

Article Structure

Most readers read articles on laptops, tablets and cell phones. The story has the picture, title, sub-title, and maybe three sentences to capture the reader’s attention. Medium keeps statistics:

  • Views — a reader opened the article (got the reader’s attention)
  • Reads — the reader read the entire article (inspired the reader to finish)
  • Fans — the reader clapped one to fifty times (the reader’s appreciation)

In the electronic world, long paragraphs turn some readers off. I find myself irritated with long paragraphs in my Kindle books, even if they are grammatically correct.

Generally, I keep paragraphs to three or four sentences, with the key concept in the first sentence or a conclusion in the last one, if possible. I have noticed the same problem with long Facebook posts. Now, I split my posts with blank lines.

Conclusion

I had a few technical articles published over my 35-year career. But after retiring, Medium has provided me an excellent forum for publishing high-quality articles on a wide range of subjects. I write whenever I want and don’t have to meet any specific deadline.

Finally, my brother will be astounded at the brevity of this article.

The quality of Medium articles tend to be at the Journalistic level, as explained in this article:

Other articles in the same category

Writing
Teaching
Médium
Life Lessons
Critique Groups
Recommended from ReadMedium