Refined RAG : Duo of Merge Retrieval & Long Context Reorder : Part 1
Unveiling the Power of LOTR & LCR in Retrieval-Augmented Generation
In Part 2, we will see code example of all concepts we discussed in this part
Retrieval-Augmented Generation (RAG) models have been a significant breakthrough in the field of Natural Language Processing (NLP). They combine the best of both worlds :
- The power of pre-trained language models
- And the ability to use external knowledge from a retriever model.
We will run a series of articles, where we will delve into an advanced version of RAG, which we call “Refined RAG”. This part introduces two key enhancements: Merge Retrieval and Long Context Reorder. in part 2 we will see a particle example.
Merge Retrieval
What is it ?
Merge retrieval refers to the process of combining information from multiple data storage areas, known as VectorStores, using a tool feature called Merge Retriever. I think it’s humorously named “LOTR: Lord of the Retrievers” :) . The primary objective is to amalgamate data retrieved from different sources and present a unified list of relevant documents.
What is the purpose ?
The use of merge retrieval addresses several challenges associated with relying on a single knowledge base for diverse types of documents. Here’s why it is beneficial:
- Avoiding Confusion : Using a single knowledge base for various document types can lead to confusion for retrieval models. Retrieval models may struggle to discern the appropriate information based on the specific topic or context of the query. By utilizing merge retrieval, data from distinct VectorStores is combined, ensuring a more comprehensive and contextually relevant set of results.
- Organizing Information Effectively : Multiple VectorStores are created as separate data storage areas for different document types. This organizational structure allows for a more systematic and efficient arrangement of information. Each VectorStore can be optimized and tailored to handle specific document categories, improving overall system organization.
- Enhancing Relevance : Merge retrieval contributes to enhancing the relevance of the retrieved information. By consolidating results from various sources, it ensures that the final list of documents presented to the user is comprehensive and well-ranked. This can lead to a more accurate and satisfactory user experience in information retrieval systems.
What is real world use case
Let’s consider a real-world example. Suppose you’re using a RAG model to answer the question, “What are the health benefits of green tea?” In the first retrieval step, the model might retrieve documents about tea in general, some of which might not be specifically about green tea. After generating a refined query like “health benefits of green tea”, the model retrieves a second set of documents specifically about green tea. The final set of documents, which is a merge of the two sets, provides a more comprehensive answer to the question.
Long Context Reorder
What is it ?
In the realm of natural language processing and machine learning, the concept of “Long Context Reorder” emerges as a pivotal technique aimed at refining how models allocate attention across the entirety of textual data. The fundamental objective is to encourage models to distribute their attention uniformly, avoiding biases toward the initial or concluding segments of the text. The importance of Long Context Reorder lies in rectifying the inherent tendency of models to predominantly focus on the beginning or end of a given text(aka ‘Lost in the middle”). This bias may result in overlooking crucial information nestled within the middle sections, leading to suboptimal understanding and response generation.
What is the purpose ?
The purpose of Long Context Reorder (LCR) or Long Context Re-ranking in Retrieval-Augmented Generation (RAG) is to improve the accuracy and performance of the model by reordering the retrieved documents based on their similarity score with the user’s query. The traditional method of ranking documents in descending order of their vector similarity score might not be the best approach, especially when dealing with more than 10 documents. When Long Context Reorder is applied, the most similar documents are placed at the top, the least similar ones are placed at the bottom, and the documents with intermediate similarity scores are placed in the middle. This approach ensures that the documents containing the correct answer are placed near the top or bottom of the context, which improves the accuracy of the model.
What is real world use case?
Consider a customer support system where machine learning models are employed to analyze and respond to customer support tickets. In this scenario, implementing Long Context Reorder becomes particularly valuable to ensure comprehensive understanding and accurate responses.
Customer support tickets often contain vital information distributed throughout the entire text. However, due to the conventional bias of models towards the beginning or end of a message, there is a risk of overlooking crucial details buried within the middle sections of the ticket. This could lead to incomplete comprehension of customer concerns and result in suboptimal responses.
By incorporating Long Context Reorder techniques, the model is trained to pay equal attention to all parts of the customer support ticket. This ensures that the model doesn’t favor the initial or concluding sections, enabling it to capture nuanced details and context embedded in the middle of the text.
