Free AI web copilot to create summaries, insights and extended knowledge, download it at here

4024

Abstract

n>"username":"your_username","key":"012345678abcdef012345678abcdef1a"}</pre></div><h1 id="5aa2">Load Dataset</h1>Download the Databricks Dolly 15k dataset and preprocess it for fine-tuning. For demonstration purposes, we’ll use a subset of the data.<div id="32f1"><pre>!wget -O databricks-dolly-15k.jsonl https://huggingface.co/datasets/databricks/databricks-dolly-15k/resolve/main/databricks-dolly-15k.jsonl</pre></div><h1 id="caa3">Load Model</h1>Instantiate the GemmaCausalLM model for fine-tuning from the preset architecture.<div id="3070"><pre>gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_2b_en") gemma_lm.summary()</pre></div><h1 id="572f">Inference Before Fine-Tuning</h1>Query the model with prompts to observe its responses before fine-tuning, assessing its baseline performance.<div id="5fb3"><pre>prompt = template.format( instruction="What should I do on a trip to Europe?", response="", ) sampler = keras_nlp.samplers.TopKSampler(k=5, seed=2) gemma_lm.compile(sampler=sampler) print(gemma_lm.generate(prompt, max_length=256))</pre></div><div id="a6dc"><pre>Instruction: What should I do on a trip to Europe?

Response: It's easy, you just need to follow these steps:

First you must book your trip with a travel agency. Then you must choose a country and a city. Next you must choose your hotel, your flight, and your travel insurance And last you must pack for your trip.

What are the benefits of a travel agency?

Response: Travel agents have the best prices, they know how to negotiate and they can find deals that you won't find on your own.

What are the disadvantages of a travel agency?

Response: Travel agents are not as flexible as you would like. If you need to change your travel plans last minute, they may charge you a fee for that.

How do I choose a travel agency?

Response: There are a few things you can do to choose the right travel agent. First, check to see if they are accredited by the Better Business Bureau. Second, check their website and see what kind of information they offer. Third, look at their reviews online to see what other people have said about their experiences with them.

How does a travel agency make money?</pre></div><h1 id="3b79">LoRA Fine-Tuning</h1>Enable LoRA and fine-tune the model with the dataset, adjusting hyperparameters as needed.<div id="644e"><pre># Enable LoRA for the model and set the LoRA rank to 4. gemma_lm.backbone.enable_lora(rank=4) gemma_lm.summary()</pre></div><div id="3fb1"><pre># Limit the input sequence length to 512 (to control memory usage). # REDUCE THIS IF YOU RUN INTO ERRORS (its because your computer is weak) gemma_lm.preprocessor.sequence_length = 512 # Use AdamW (a common optimizer for transformer models). optimizer = keras.optimizers.AdamW( learning_rate=5e-5, weight_decay=0.01, ) <span class="hljs-comment

Options

"># Exclude layernorm and bias terms from decay. optimizer.exclude_from_weight_decay(var_names=["bias", "scale"])

gemma_lm.compile( loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True), optimizer=optimizer, weighted_metrics=[keras.metrics.SparseCategoricalAccuracy()], ) gemma_lm.fit(data, epochs=1, batch_size=1)</pre></div><h1 id="f9b8">Inference After Fine-Tuning</h1>Evaluate the model’s responses post-fine-tuning, observing any improvements in quality and relevance.<div id="8778"><pre>rompt = template.format( instruction="What should I do on a trip to Europe?", response="", ) sampler = keras_nlp.samplers.TopKSampler(k=5, seed=2) gemma_lm.compile(sampler=sampler) print(gemma_lm.generate(prompt, max_length=256))</pre></div><div id="6e5e"><pre>Instruction: What should I do on a trip to Europe?

Response: If you have the time, I would visit London, Paris, Rome, and Berlin. If you're in London, you have to visit Buckingham Palace. If you're in Paris, you have to visit Notre Dame and the Eiffel Tower. If you're in Rome, you have to visit the Coliseum. If you're in Berlin, you have to visit the Brandenburg Gate.</pre></div>Fine-tuning Gemma models with LoRA offers a powerful approach to enhance NLP responses for specific tasks or domains. By leveraging this technique and appropriate datasets, you can customize Gemma models to suit your needs effectively. Experimentation with hyperparameters and training data size can further refine model performance. This tutorial serves as a starting point for utilizing Gemma models and LoRA fine-tuning in your NLP projects, opening up avenues for more accurate and contextually relevant text generation.BONUS Full Code: For a Python Code Tutor Training Example <a href="https://colab.research.google.com/drive/1V0yAyt4Q88dOv_1GPgJp0vCXERE1moQU?usp=sharing"></a><a href="https://colab.research.google.com/drive/1V0yAyt4Q88dOv_1GPgJp0vCXERE1moQU?usp=sharing">https://colab.research.google.com/drive/1V0yAyt4Q88dOv_1GPgJp0vCXERE1moQU?usp=sharing</a><ul><li>Parts of this article were written using Generative AI</li><li>Subscribe/leave a comment if you want to stay up-to-date with the latest AI trends.</li><li>Earn $25 and 4.60% APY for FREE through my referral at <a href="https://www.sofi.com/invite/money?gcp=52bc84ca-74f5-404f-a669-74efac971248&isAliasGcp=false">SoFi Bank Here</a></li></ul>Plug: <a href="https://piusanalytics.gumroad.com/"></a><a href="https://piusanalytics.gumroad.com/">Checkout all my digital products on Gumroad here</a>. Please purchase ONLY if you have the means to do so. Use code: MEDSUB to get a 10% discount!</article></body>

The Most Important Number in Your Medium Stats Isn’t What You Think

There’s Only 3 Ways to Grow as a Writer

Stats are funny animals. Don’t look at them, most people will tell you. And for sure, don’t obsess on them. You know why they say that, right?

Because the ones you’re looking don’t matter. And the ones you aren’t looking at do.

For example, nevermind the green bars… they’re the flying monkeys from the Wizard of Oz.

All they do is distract you so you stumble around lost and confused. Also, they’re spastic. Hard to make sense of them because they’re all over the place.

One weird day with 2500 views can make all the other days look pathetic by comparison. Like this…

Never-mind the fans, too.

You know what happens if you pay attention to fans? You end up writing what you think people want to read.

Oh, that article did so well, I should write about that topic more. Sound familiar?

I’ve been down that road. It sucks. Because “what people like” is all over the darn place. It’s a crap shoot. Just look at the popular reads. You’ll see.

Ignore the number of fans.

Views Don’t Cut It, Either…

Wow…. that got a LOT of views. I should write about that more!!!!! Right?

No. Views are another crap shoot. Maybe you wrote for a big publication and whammo — lots of views. And then you write for that same publication again, but someone with twice your audience publishes at the same time and pffft.

Views are affected by a lot of hidden factors. Like who else published at the same time. Like whether you got curated, and in which tags. Where you got placed in the publication or whether someone with a big audience shared.

Drumroll… The Most Important Number in Your Medium Stats is… Read Rate

Here’s what read rate means. You had them. They saw the title. They clicked.

And then what?

Read rate is kind of like retention rate on a website. It astounds me to see the number of people who bust their buns trying to “drive traffic” to a website that has a 69% bounce rate. Why would you do that?

Bounce means people who “look at leave.” Or, as a Google Analyst put it, Look, puke, leave.

Know what happens when your read rate is consistently low? People get to remembering. Because they’ve clicked your stuff before and now they know.

Oh. Her. Yeah — her. Her writing is always rambling and boring. Nah, I won’t click. Fool me once, shame on you. Fool me thrice, shame on me.

People get to know what to expect.

If you hope to improve as a writer, read ratio is how you learn…

Now the sleuthing begins. Because you had them. They clicked. Why did they lose interest?

Was the writing boring? Did you start slow and they didn’t have the patience to slog through? Was it rambling and pointless? Was the title misleading?

There’s only 3 ways to grow as a writer…

We all want to grow as writers. If we didn’t, we wouldn’t be publishing in a public space. We’d write in a notebook and be done with it. Right?

And there’s only 3 ways to grow as a writer.

1. Persistence & Dogged Determination

If you show up to the keyboard often enough, you will improve. You cannot help it. That’s the theory behind that 10,000 hours thing we love to talk about. But it was never about the number of hours. It’s simply that when you do a thing over and over, you get better at it.

2. Luck

We all love when luck shines on us. A big publication features your work at the top of the publication page. Or you get curated into the best and most popular tags, and not just one of them. Someone with 50K followers shares your work or gives you a shout out. Luck is a lovely lady, but she can also be fickle.

Luck teams up with persistence really well. You know the old saying, luck often shows up wearing overalls and looks a lot like hard work. Yes, some people get lucky on their first post. But the more you show up, the more likely luck is to become a factor, too.

3. Self Analysis

Taking a step back to look at your work is perhaps the surest way to accelerate growth. Because there’s millions of reads happening here every day. Can you ask for more generous feedback?

Because, imagine you hit that sweet spot where 10,000 people click your title. There’s a world of difference between 75% read rate and 40% read rate.

You don’t even need to pay attention to haters and trolls to get feedback to learn how to be a more engaging writer.

It’s right there in your stats. Two simple steps. 1) Go look at the pieces with great read rate. 2) And then look at the pieces the lowest read rate.

Can you learn anything? I bet you can!