avatarAgu Sergius Alex, MD ✨

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

4024

Abstract

n><span class="hljs-attr">"username"</span><span class="hljs-punctuation">:</span><span class="hljs-string">"your_username"</span><span class="hljs-punctuation">,</span><span class="hljs-attr">"key"</span><span class="hljs-punctuation">:</span><span class="hljs-string">"012345678abcdef012345678abcdef1a"</span><span class="hljs-punctuation">}</span></pre></div><h1 id="5aa2">Load Dataset</h1><p id="b5cb">Download the Databricks Dolly 15k dataset and preprocess it for fine-tuning. For demonstration purposes, we’ll use a subset of the data.</p><div id="32f1"><pre>!wget -O databricks-dolly-15k.jsonl https://huggingface.co/datasets/databricks/databricks-dolly-15k/resolve/main/databricks-dolly-15k.jsonl</pre></div><h1 id="caa3">Load Model</h1><p id="4aef">Instantiate the GemmaCausalLM model for fine-tuning from the preset architecture.</p><div id="3070"><pre>gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset(<span class="hljs-string">"gemma_2b_en"</span>) gemma_lm.summary()</pre></div><h1 id="572f">Inference Before Fine-Tuning</h1><p id="45ea">Query the model with prompts to observe its responses before fine-tuning, assessing its baseline performance.</p><div id="5fb3"><pre>prompt = template.<span class="hljs-built_in">format</span>( instruction=<span class="hljs-string">"What should I do on a trip to Europe?"</span>, response=<span class="hljs-string">""</span>, ) sampler = keras_nlp.samplers.TopKSampler(k=<span class="hljs-number">5</span>, seed=<span class="hljs-number">2</span>) gemma_lm.<span class="hljs-built_in">compile</span>(sampler=sampler) <span class="hljs-built_in">print</span>(gemma_lm.generate(prompt, max_length=<span class="hljs-number">256</span>))</pre></div><div id="a6dc"><pre>Instruction: What should I do on a trip to Europe?

Response: It's easy, you just need to follow these steps:

First you must book your trip with a travel agency. Then you must choose a country and a city. Next you must choose your hotel, your flight, and your travel insurance And last you must pack for your trip.

What are the benefits of a travel agency?

Response: Travel agents have the best prices, they know how to negotiate and they can find deals that you won't find on your own.

What are the disadvantages of a travel agency?

Response: Travel agents are not as flexible as you would like. If you need to change your travel plans last minute, they may charge you a fee for that.

How do I choose a travel agency?

Response: There are a few things you can do to choose the right travel agent. First, check to see if they are accredited by the Better Business Bureau. Second, check their website and see what kind of information they offer. Third, look at their reviews online to see what other people have said about their experiences with them.

How does a travel agency make money?</pre></div><h1 id="3b79">LoRA Fine-Tuning</h1><p id="9bc6">Enable LoRA and fine-tune the model with the dataset, adjusting hyperparameters as needed.</p><div id="644e"><pre><span class="hljs-comment"># Enable LoRA for the model and set the LoRA rank to 4.</span> gemma_lm.backbone.enable_lora(rank=<span class="hljs-number">4</span>) gemma_lm.summary()</pre></div><div id="3fb1"><pre><span class="hljs-comment"># Limit the input sequence length to 512 (to control memory usage).</span> <span class="hljs-comment"># REDUCE THIS IF YOU RUN INTO ERRORS (its because your computer is weak)</span> gemma_lm.preprocessor.sequence_length <span class="hljs-punctuation">=</span> <span class="hljs-number">512</span> <span class="hljs-comment"># Use AdamW (a common optimizer for transformer models).</span> optimizer <span class="hljs-punctuation">=</span> keras.optimizers.AdamW<span class="hljs-punctuation">(</span> learning_rate<span class="hljs-punctuation">=</span><span class="hljs-number">5</span>e-<span class="hljs-number">5</span>, weight_decay<span class="hljs-punctuation">=</span><span class="hljs-number">0.01</span>, <span class="hljs-punctuation">)</span> <span class="hljs-comment

Options

"># Exclude layernorm and bias terms from decay.</span> optimizer.exclude_from_weight_decay<span class="hljs-punctuation">(</span>var_names<span class="hljs-punctuation">=</span><span class="hljs-punctuation">[</span><span class="hljs-string">"bias"</span>, <span class="hljs-string">"scale"</span><span class="hljs-punctuation">]</span><span class="hljs-punctuation">)</span>

gemma_lm.compile<span class="hljs-punctuation">(</span> loss<span class="hljs-punctuation">=</span>keras.losses.SparseCategoricalCrossentropy<span class="hljs-punctuation">(</span>from_logits<span class="hljs-punctuation">=</span><span class="hljs-literal">True</span><span class="hljs-punctuation">)</span>, optimizer<span class="hljs-punctuation">=</span>optimizer, weighted_metrics<span class="hljs-punctuation">=</span><span class="hljs-punctuation">[</span>keras.metrics.SparseCategoricalAccuracy<span class="hljs-punctuation">(</span><span class="hljs-punctuation">)</span><span class="hljs-punctuation">]</span>, <span class="hljs-punctuation">)</span> gemma_lm.fit<span class="hljs-punctuation">(</span>data, epochs<span class="hljs-punctuation">=</span><span class="hljs-number">1</span>, batch_size<span class="hljs-punctuation">=</span><span class="hljs-number">1</span><span class="hljs-punctuation">)</span></pre></div><h1 id="f9b8">Inference After Fine-Tuning</h1><p id="2c82">Evaluate the model’s responses post-fine-tuning, observing any improvements in quality and relevance.</p><div id="8778"><pre>rompt = template.<span class="hljs-built_in">format</span>( instruction=<span class="hljs-string">"What should I do on a trip to Europe?"</span>, response=<span class="hljs-string">""</span>, ) sampler = keras_nlp.samplers.TopKSampler(k=<span class="hljs-number">5</span>, seed=<span class="hljs-number">2</span>) gemma_lm.<span class="hljs-built_in">compile</span>(sampler=sampler) <span class="hljs-built_in">print</span>(gemma_lm.generate(prompt, max_length=<span class="hljs-number">256</span>))</pre></div><div id="6e5e"><pre>Instruction: What should I do on a trip to Europe?

Response: If you have the time, I would visit London, Paris, Rome, and Berlin. If you're in London, you have to visit Buckingham Palace. If you're in Paris, you have to visit Notre Dame and the Eiffel Tower. If you're in Rome, you have to visit the Coliseum. If you're in Berlin, you have to visit the Brandenburg Gate.</pre></div><p id="3274">Fine-tuning Gemma models with LoRA offers a powerful approach to enhance NLP responses for specific tasks or domains. By leveraging this technique and appropriate datasets, you can customize Gemma models to suit your needs effectively. Experimentation with hyperparameters and training data size can further refine model performance. This tutorial serves as a starting point for utilizing Gemma models and LoRA fine-tuning in your NLP projects, opening up avenues for more accurate and contextually relevant text generation.</p><p id="7fce"><b>BONUS Full Code: For a Python Code Tutor Training Example <a href="https://colab.research.google.com/drive/1V0yAyt4Q88dOv_1GPgJp0vCXERE1moQU?usp=sharing"></a></b><a href="https://colab.research.google.com/drive/1V0yAyt4Q88dOv_1GPgJp0vCXERE1moQU?usp=sharing">https://colab.research.google.com/drive/1V0yAyt4Q88dOv_1GPgJp0vCXERE1moQU?usp=sharing</a></p><ul><li><i>Parts of this article were written using Generative AI</i></li><li>Subscribe/leave a comment if you want to stay up-to-date with the latest AI trends.</li><li>Earn <b>$25</b> and <b>4.60% APY</b> for <b>FREE</b> through my referral at <a href="https://www.sofi.com/invite/money?gcp=52bc84ca-74f5-404f-a669-74efac971248&amp;isAliasGcp=false">SoFi Bank Here</a></li></ul><p id="ebf0"><b><i>Plug: <a href="https://piusanalytics.gumroad.com/"></a></i></b><i><a href="https://piusanalytics.gumroad.com/">Checkout all my digital products on Gumroad here</a>.<b> </b>Please purchase ONLY if you have the means to do so. Use code: MEDSUB to get a 10% discount!</i></p></article></body>

7 Habits of Highly Effective Medium Writers

How to Improve Your Writing Skills and Grow Your Following on Medium

Photo by Nubelson Fernandes on Unsplash

As a Medium writer, developing habits that can help you become more effective in your writing and increase the chances of your articles reaching a wider audience is crucial. I will tell you what I have learned from writing on Medium in the last two months. I have put everything together in this writing titled, “The Seven (7) Habits of Highly Effective Medium Writers” which has helped me improve my writing skills and grow my following.

Writing Every Day

When I first started writing on Medium, I wasn’t sure who my audience was or why I should write every day. It felt like I was writing a letter and posting it publicly, hoping that someone would stumble upon it and find it interesting enough to read, clap, and comment on. However, after reading advice from successful and highly effective Medium writers, I realized that developing a daily writing habit was crucial.

Whether it’s a short post or a lengthy article, writing regularly helps to build skills and improve writing style. I’ve had days where I’ve written three articles and others where I couldn’t write at all, but I still made sure to have something to account for every passing day in the last two months. On average, I spend an hour or more writing daily, often while reading at the same time.

Research Thoroughly

Before writing an article, I try to research the topic thoroughly. This has helped me to gather relevant information and insights to use to create a high-quality article. To me, the spotlight is always the title and the subtitle because I feel that if I get it right, then every other thing will follow. Research can also help you to identify what has been written on the topic and whether you have anything to add or what perspective to write from, whether in support or against the idea. Research gives credence to your writing and gives your readers something original, impactful, and something for their time.

Develop Your Voice

I have realized that the best way to gain a writing voice is to write from your viewpoint. Start by telling your personal story; this is one thing I have discovered that many people do on Medium — writing about their writing. Your writing voice is what sets you apart from other writers. Apart from telling your personal story, you can also experiment with other writing styles until you find one that suits your personality and resonates with your readers. I am still a learner when it comes to finding my voice as a writer, but I can say that I have made a big improvement.

Edit Ruthlessly

Editing allows you to refine your writing and make it more concise and engaging. It removes repetition and cuts out any unnecessary words, sentences, or paragraphs. Look for ways to make your writing more clear and direct. I use the free version of Grammarly, which I am thinking of upgrading.

Engage With Your Audience

Writing is a form of communication, and highly effective Medium writers understand the importance of engaging with their audience. This will involve responding to comments on your articles, participating in discussions, and reaching out to readers who have clapped and commented on your writing.

Set Goals

Setting goals has helped me stay focused and motivated and gives me something to strive for, like knowing that there has to be at least one article to account for every passing day. A writer’s goals have to be specific, measurable, achievable, relevant, and time-bound.

Learn Continuously

I have also learned that highly effective Medium writers are always learning. They read other writers’ work, buy books relevant to the niche, and study the craft of writing. Learning helps you to stay up-to-date with the latest trends and techniques in your niche and can help you to improve your skills over time.

Finally, I have looked at my stats on Medium and come to a conclusion. The seven habits of highly effective Medium writers are geared towards writing more consistently and embracing the conversational aspect of your writing. It is believed that if you write consistently for one month, by a year of incorporating these habits into your writing routine, you will become a better writer with content that resonates with your readers. Start practicing these habits today and take your writing to the next level.

Habits
Writing
Medium
How To
Followers
Recommended from ReadMedium
avatarEve Arnold
Just Write

The best advice

4 min read