Summary

The article discusses the application of Maximal Marginal Relevance (MMR) to improve the diversity and relevance of keyphrases extracted from documents, addressing the limitation of traditional methods that often select similar and redundant phrases.

Abstract

The article titled "Maximal Marginal Relevance to Re-rank results in Unsupervised KeyPhrase Extraction" explores the use of MMR, a technique originally introduced for document summarization, to enhance the unsupervised keyphrase extraction process. The author highlights the issue with conventional keyphrase extraction methods, which tend to rank similar phrases highly, leading to redundancy in the top results. To illustrate the problem, the author provides an example where keyphrases like "Good Product," "Great Product," and "Nice Product" dominate the top rankings, despite conveying similar sentiments. The article presents two approaches to mitigate this issue: the first involves using cosine similarity to identify and remove redundant phrases, which requires setting a similarity threshold that can be challenging to calibrate. The second, more effective approach employs MMR to re-rank keyphrases by balancing query relevance with the novelty of information, thus ensuring diversity in the final selection of keyphrases. The author provides code snippets to demonstrate the implementation of both methods and concludes that MMR, with an optimally set lambda parameter, successfully addresses the redundancy issue by spreading out similar phrases in the ranking, thereby allowing for a more informative and diverse set of top keyphrases.

Opinions

The author believes that traditional keyphrase extraction methods, such as TextRank and RAKE, can lead to redundancy in the extracted keyphrases.
The author considers the use of cosine similarity to remove redundant phrases as a naive approach that requires manual threshold tuning, which can be problematic.
The author endorses MMR as a superior method for re-ranking keyphrases, as it balances the relevance to the query with the diversity of the results, thus providing a better representation of the document's content.
The author suggests that setting the lambda value in the MMR formula to 0.5 offers an optimal balance between diversity and accuracy in the keyphrase selection process.
The author invites feedback and discussions on the topic, indicating an openness to collaborative improvement and a willingness to engage with the professional community on LinkedIn.

Maximal Marginal Relevance to Re-rank results in Unsupervised KeyPhrase Extraction

Maximal Marginal Relevance a.k.a. MMR has been introduced in the paper The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. MMR tries to reduce the redundancy of results while at the same time maintaining query relevance of results for already ranked documents/phrases etc.

We first try to understand the scenario by taking an example and will see how MMR is helpful in solving the issue.

Recently I was trying to extract KeyPhrases from a set of documents that belongs to one category. I have used different approaches (TextRank, RAKE, POS tagging, etc.. to name a few) to extract keywords from the documents, which provides phrases along with score. This score is used as the ranking of the phrases for that document.

Let’s say your final keyPhrases are ranked like Good Product, Great Product, Nice Product, Excellent Product, Easy Install, Nice UI, Light weight etc. But there is an issue with this approach, all the phrases like good product, nice product, excellent product are similar and define the same property of the product and are ranked higher. Suppose we have a space to show just 5 keyPhrases, in that case, we don't want to show all these similar phrases.

You want to properly utilize this limited space such that the information displayed by the Keyphrases about the documents is diverse enough. Similar types of phrases should not dominate the whole space and users can see a variety of information about the document.

We are going to address this problem in this blog post. There might be different approaches to solve this problem. For the sake of simplicity and completeness of the article, I am going to discuss two approaches:

Remove redundant phrases using cosine similarity:

To use cosine similarity is the naive approach that came to mind to deal with terms having the same meaning. Use word embeddings to find embeddings of phrases and find cosine similarity between embeddings. Set a threshold above which you will consider the terms as similar. Just take one keyPhrase having more score out of clubbed phrases in the result.

An issue with this approach is that you need to set the threshold (0.9 in code) above which, terms will be clubbed together. And sometimes very close keywords might have cosine similarity < threshold. Word embeddings have been used to convert the sentence to vector by averaging word tokens. Keeping the threshold low will lead to dealing with the same issue again. I find it difficult to manually tweaking this threshold to include all edge cases.

2. Re-Ranking the KeyPhrases using MMR

The idea behind using MMR is that it tries to reduce redundancy and increase diversity in the result and is used in text summarization. MMR selects the phrase in the final keyphrases list according to a combined criterion of query relevance and novelty of information.

The latter measures the degree of dissimilarity between the document being considered and previously selected ones already in the ranked list. [1]

MMR ranking provides a useful way to present information to the user that is not redundant. It considers the similarity of keyphrase with the document, along with the similarity of already selected phrases.

where, Q = Query (Description of Document category)
 D = Set of documents related to Query Q 
 S = Subset of documents in R already selected 
 R\S = set of unselected documents in R 
 λ = Constant in range [0–1], for diversification of results

In the below implementation of MMR, cosine similarity has been considered as Sim_1 and Sim_2. Any other similarity measure can be taken and the function can be modified accordingly.

Setting λ to 0.5 gives the optimal mix of diversity and accuracy in the result set. The value of λ can be set based on the use-case and your dataset.

MMR helps to address the issue by ranking similar phrases far away. So the issue to select top N keyPhrase has been resolved as all similar terms are not grouped and don’t appear in the final result.

Please let me know if you like the post, or have some suggestions/concerns and feel free to reach out to me on LinkedIn.

References: