Why Graph Based RAG Is The Future?
You need to ground LLMs w/ Graphs Not Vectors for Enterprise AI, next part we will build one !
Businesses are more and more using Generative AI to use what they know and the information they have. But just using language models by themselves might not give the full picture of what’s going on inside the company. Why — These models are taught using information from public sources, so they might not fully get company’s unique details.
The Solution? RAG (Retrieval-Augmented Generation) !
In a very simple form, RAG involves selecting the appropriate data (from internal company store) to respond to a question and providing it to the large language model (LLM). Subsequently, the LLM can analyze the data and produce a response.
How we are doing this so far?
We ( I've been doing this for past 10+ months now) handle RAG by doing a vector retrieval, from vector stores/ databases. It’s for sure a very nice approach, and I liked it, but after months of experiments I realized certain issues. Let’s explore them.
A quick look at how we do RAG w/ Vector retrieval
Process : In this step, information such as text undergoes a process where it’s broken down into smaller segments, typically 200 characters each. These segments are then transformed into numerical representations known as vector embeddings. These embeddings are stored in a special type of database called a vector database.
The connections between different pieces of information are established by measuring the numerical differences between their vector embeddings. For instance, in a search for related concepts like “dog” and “cat,” these terms would be closer to each other numerically compared to “dog” and “car”
Retrieval : When a user submits a query or a question, the query itself is also transformed into a vector embedding. Using algorithms such as K-Nearest Neighbors (KNN) or Approximate Nearest Neighbors (ANN), the system identifies and retrieves the closest data points (“k”) to the query from the vector database.
Generation : Finally, the top “k” vector embeddings, representing the most relevant data segments, are translated back into their original forms. These data segments are then passed on to the large language model (LLM), which uses its reasoning capabilities to generate an answer. For example, if the query is about historical events, the LLM may use the retrieved data to generate a coherent explanation or narrative.
So, what’s an issue ?
Below is based on my learning, and there may be good techniques to overcome few of it, but I firmly believe there’s a better way ( we will see in a bit) even after using advanced technique in RAG using vectors
Below is my try to put in very simpler form w/o going into dense technical details like “dimensionality curse”, sensitivity to noise and outliers, KNN scalability issues, optimal ‘k’ etc ( hope is easier to grasp issues)
Vector-based search is a way to find information, but it has some big problems. Here’s what you need to know:
- Chopping Things Up: Computers don’t understand things the way we do. To use vectors, they break text into small pieces. This can lose important connections between words and make the results less useful.
- Old-Fashioned Math: Vector-based search uses old-school math tricks to find similar stuff. These tricks don’t work well with big amounts of information or when the information is complex. This makes it slow and can lead to wrong answers.
- Numbers vs. Words: Vector-based search struggles when you have both text and numbers. Think of looking for answers in a manual alongside a spreadsheet. The computer gets the types of information mixed up.
- Pricey and Difficult: This kind of search needs expensive tools that are hard to manage. Every time you add information, the computer has to redo a lot of work, making it slow and costly.
So what’s the future for RAG ? * Graphs *
Here’s how I think RAG (Retrieval Augmented Generation) using a graph-based approach, and how it addresses the limitations of vector-based retrieval while paving the way for the future: ( we will build it in next part from scratch)
Imagine you have a large database of information about movies — including details like the plot, characters, actors, directors, genres, reviews, box office numbers etc. This diverse information exists in different formats like text files, pdfs, spreadsheets and videos.
In a graph-based approach, each movie would be represented as a node in the graph. Edges would then connect each movie node to other nodes representing the related data — like nodes for the director, actors, genres, plot points and so on. The edges define the semantic relationships between the entities.
Now if you ask a question like “What are some highly rated superhero movies directed by female filmmakers?”, the natural language processing component understands the key criteria — genre is superhero, director gender is female, and we want highly rated movies.
The retrieval algorithms then traverse the graph, identifying the relevant movie nodes that have genre=”superhero”, are connected to director nodes with gender=”female”, and also look at rating/review nodes to filter for highly rated films.
This contextual retrieval from the graph gives the language model a rich set of inter-linked data points like the movie titles, plots, cast/crew details and reviews of the relevant films to then generate an accurate and comprehensive response.
Compare this to vector-based retrieval which would simply retrieve the closest matching documents/chunks based on semantic similarity of the words/phrases, lacking the contextual connections and relationships.
The graph model’s strengths really shine for diverse enterprise datasets containing multimodal data. You can simply add new data entities as nodes and define their relationships, without reprocessing everything. This flexibility and efficiency makes it highly scalable.
So what are our take aways
RAG with Graphs: A Better Way to Search
- The Smart Map : Imagine your information as a giant map. Each important idea is a place on the map, and lines between places show how they’re connected. A customer on a shopping website might be connected to things they bought, things they looked at, and their previous orders. This map is called a graph.
- Finding Your Way: When you ask a question, the graph helps find the best answers. It understands the meaning of your question and knows how the information on the map is linked. This means it can find things that are truly related, not just things with similar words.
Why Graphs Fix the Problems
- No More Chopping: With graphs, information stays connected. No random cutting into tiny pieces means the computer keeps the context and understands the bigger picture.
- Smarter Search: Graphs use clever techniques that work well with complex information, even when there’s a lot of it. This means faster, more accurate results.
- Flexible Data: Graphs don’t mind if your information is in documents, spreadsheets, videos, or anything else. They can handle it all.
- Easy Updates: Got new information? Just add it to your map! No need for the computer to recalculate everything like with vector-based systems. This saves time and money.
Why Graphs Are the Future
Graphs help computers understand information more like people do. This leads to more helpful answers and opens up new possibilities for smart systems that can really work with our knowledge.
