Large Language Models — Pre-Training and Fine-Tuning

Fundamentals of Large Language Models and their down-stream tasks.

1. Intro

As an applied scientist working in the Natural Language Processing (NLP) domain, I use Pre-Trained Models (PTMs) on a daily basis, one type of which are Large Language Models (LLMs). We all use LLMs on a daily basis in our lives without even knowing it. For example, when you talk to your favorite virtual assistant (e.g. Alexa, Siri, Google), or when you use Google Translate, these engines rely on LLMs one way or another. Despite the increased public attention to the domain of LLMs, I was having a hard time finding a brief primer on what PTMs are in NLP so I created this post. In this post, I will provide an overview of what LLMs are, will explain what we actually mean by “pre-training” and also will provide some examples of the steps usually involved in creating LLMs, which include pre-training and fine-tuning for downstream tasks (e.g. machine translation, question answering, named-entity recognition, classification, etc.).

2. What Are Large Language Models?

Large language models (LLMs), in the context of NLP, are models trained on large textual data, without knowing how to do any specific downstream tasks — this step is called “Pre-Training”. The idea is that the model will see large amounts of data and then will glean some “general knowledge” from the data. This is in contrast to traditional machine learning techniques where we usually teach the model what to do. During “pre-training”, we just let the model gain some general knowledge and then once we have the pre-trained model, we can teach it to perform certain tasks. The last step where we teach the LLM to perform specific tasks is called “Fine-Tuning”.

An analogy might help with the understanding of “pre-training” and “fine-tuning” steps. Let’s assume an individual that does not know anything in the beginning (I know this is not realistic but let’s continue with the assumption). Then we provide the individual with lots of books in English and with as many books in French but we do not tell the individual anything. This individual is then surprised to find out that he somehow can read these books and as a result by the end of this process gains a “general knowledge” of both English and French from the books the has studied — this is equivalent to “pre-training” a large language model. We need to make another unrealistic assumption here — this individual has a very strong memory so he retains a lot of this information about English and French books that he has studied. Then we start teaching the individual using a smaller data set that includes English sentences followed by their French translations — this is equivalent to the “fine-tuning” stage of a model. Remember that so far this individual did not know that languages could be translated to each other but he does have a very good memory so he has a general knowledge about each language separately. After the individual sees these examples of English sentences and their French translations, the individual now understands that other parts of his French knowledge are also the translations of his English knowledge and therefore this individual can now translate English to French overall. This sudden realization in LLMs is quite powerful in that by seeing smaller data sets that show the LLMs what the task is (translation in our example), the model suddenly realizes that it can generalize its knowledge based on those limited examples and suddenly knows how to translate English to French!

Figure below formalizes this process. A large amount of data (similar to the books in our example), denoted as 1 in the figure, is added to an untrained model (denoted as 2 in the figure), which concludes the “pre-training” step and creates an LLM. Then we want to teach the LLM a specific task to perform (denoted as 3 in the figure). We will discuss a variety of tasks but in the previous example the task was translation of English to French, which concludes the “fine-tuning” step. As a result, we now have a “fine-tuned” model that can perform the task that we have taught it to do. In the example above, this is the point where the individual is able to generate French translations of English sentences. Numbers 4 and 5 demonstrate that once we have a fine-tuned model, we can send in an input (English sentences in our example) and then the fine-tuned model will generate an output (French translation in our example).

At this point you may be wondering how this is any different from any other ML training process. In order to answer that question, let us do a refresher on the types of training methodologies normally used in ML. If you are already familiar with these, feel free to skip to the next section.

In general there are three types of training in ML as follows:

1. Supervised: When training data in labeled. Labeled data can vary based on the task that the ML model is being trained to do but as an example, in the context of machine translation, a labeled training data will include a source in the original language and its translation in the target language.
2. Unsupervised: This is a training process where the training data is not labeled. To continue with the machine translation example, unlabeled data would be when only monolingual textual data are available (i.e. translation is no longer available). This is similar to our hypothesized individual who only had access to monolingual books in English and French.
3. Self-Supervised: This is a blend of supervised and unsupervised methods where labels of the training data are generated using an automated system.

As you can tell from our walkthrough of LLMs, unlike traditional ML systems, we are using large amounts of unlabeled data (similar to unsupervised training) during the “pre-training” stage to create an LLM and then use a smaller amount of labeled data (similar to supervised training) during the “fine-tuning” stage to create our fine-tuned model, that is actually capable of performing tasks.

Next, let’s talk about why we didn’t just continue using our conventional ML models and ended up creating LLMs.

3. Why Use LLMs?

There are two main reasons that we choose to use LLMs instead of building a ML model from scratch for many tasks in NLP.

1. Data Availability: Neural networks and in particular deep learning models (these are “deeper” neural networks, meaning they include multiple hidden layers before generating an output) have been among the very promising architectures in the NLP space. These models have a large number of parameters (and this is why we call them “large” language models) and as a result require large amounts of labeled training data to avoid overfitting during training. Supervised training (which requires labeled data) is quite expensive in NLP, since it requires human intervention and therefore labeled training data is not as readily available as unlabeled data. Going back to our example, it is quite expensive to generate human translations of English to French — we would need to collect English sentences and then have human specialists translate them to French. Unlabeled data on the other hand is all around us in the internet (e.g. Wikipedia posts, etc.). Going back to our example, unlabeled data is various monolingual books and other online content that are already available in English and/or French, but not necessarily as translations of each other. As a result, NLP scientists decided to breakdown the training process into a “pre-training” step using unlabeled monolingual data (i.e. unsupervised learning), followed by a “fine-tuning” step using much smaller labeled data (i.e. supervised learning) for various downstream tasks, instead of creating ML models directly for the downstream tasks, such as machine translation.
2. Scalability (or Computational Efficiency): Since this approach breaks down the training process into two steps of “pre-training” (using unlabeled data) followed by a “fine-tuning” for downstream tasks (using smaller labeled data), it provides a more modular architecture. As a result, once an LLM is created, it can be used for multiple downstream tasks. In other words, one LLM can be fine-tuned using labeled data for each downstream task separately, generating multiple models. As an example, one LLM can be once fine-tuned for an intent classification as its downstream task and the same LLM can then be fine-tuned for named-entity recognition as another downstream task.

Figure below demonstrates how a conventional ML model defers from a pre-trained model followed by fine-tuning.

Now that we understand how and why these PTMs and LLMs exist, let’s look at what tasks we can use them for.

4. Downstream Tasks Examples

In this section I have included some of the more popular downstream tasks that LLMs can be fine-tuned on. Then, I will include a post where you can get hands-on experience with these tasks using LLMs!

1. Machine Translation: This one does not need much explanation. The engine behind Google Translate is an example of machine translation.
2. Named-Entity-Recognition: A model that identifies classes of entities in a text such as person names, locations, organization, addresses, etc. This one can be quite useful. Imagine if a company has large amounts of data and is trying to extract only location information from such data. They can leverage an LLM that is fine-tuned on named-entity recognition to identify and extract all of location information in the large data set.
3. Machine Reading Comprehension: A model that is trained to answer questions. This one is an interesting use case. For example, an LLM can be taught to read a document and then answers users’ questions, given the knowledge provided in the document.
4. Sentiment Analysis: A model trained to classify the sentiment or opinions expressed in a textual input. Usual classes include positive, negative or neutral.
5. Summarization: This one is also self-explanatory and quite popular. A model that takes a text as in input and generates a summary of the input. For example, imagine a situation where a customer reaches out to follow-up on their case with the customer representative. It is much more efficient for the customer representative to read the LLM-provided summary of that specific case history, instead of reading all the details of the case history, saving both customer and the company valuable time.

If you are interested in seeing examples of how these downstream tasks are performed and gain hands-on implementation experience of using LLMs, post below is for you!

Thanks for Reading!

If you found this post helpful, please follow me on Medium and subscribe to receive my latest posts!

(All images, unless otherwise noted, are by the author.)