LANGCHAIN — Agentic RAG with LangGraph

Software is like entropy: It is difficult to grasp, weighs nothing, and obeys the Second Law of Thermodynamics; i.e., it always increases. — Norman Augustine

LangGraph is an easy way to implement LLM state machines. Let’s focus on how we can use LangGraph for “flow engineering” of self-reflective RAG. Here, we’ll provide you with the implementation of ideas from two interesting papers, CRAG and Self-RAG, using LangGraph.

Corrective RAG (CRAG)

CRAG introduces a few interesting ideas:

1. Employ a lightweight retrieval evaluator to assess the quality of retrieved documents for a query, returning a confidence score for each.
2. Perform web-based document retrieval to supplement context if vectorstore retrieval is deemed ambiguous or irrelevant to the user query.
3. Perform knowledge refinement of retrieved documents by partitioning them into “knowledge strips”, grading each strip, and filtering out irrelevant ones.

Here’s how we can represent this in LangGraph:

from langgraph import StateMachine, Node, Edge, pydantic_model, OpenAITool

# Define the nodes
retrieval_evaluator = Node("RetrievalEvaluator")
web_search = Node("WebSearch")
query_rewrite = Node("QueryRewrite")

# Define the edges
retrieval_evaluator_to_web_search = Edge(retrieval_evaluator, web_search, condition=lambda: any_document_is_irrelevant())
web_search_to_query_rewrite = Edge(web_search, query_rewrite, condition=lambda: web_search_is_needed())

# Bind the function to the LLM
@pydantic_model
def my_function():
    return "Output"

OpenAITool.bind_function(my_function, function_name="MyFunction")

Self-RAG

The framework of Self-RAG trains an LLM to generate self-reflection tokens that govern various stages in the RAG process. Here’s how we can outline this in LangGraph:

from langgraph import StateMachine, Node, Edge, pydantic_model, OpenAITool

# Define the nodes
retrieval = Node("Retrieve")
web_search = Node("WebSearch")
generation = Node("Generation")

# Define the edges
retrieval_to_web_search = Edge(retrieval, web_search, condition=lambda: any_document_is_irrelevant())
web_search_to_generation = Edge(web_search, generation, condition=lambda: web_search_is_needed())

# Bind the function to the LLM
@pydantic_model
def my_function():
    return "Output"

OpenAITool.bind_function(my_function, function_name="MyFunction")

These examples demonstrate how easy it is to use LangGraph for “flow engineering” of self-reflective RAG. By implementing ideas from CRAG and Self-RAG, you can enhance RAG and correct poor quality retrieval or generations.

In conclusion, self-reflection can greatly enhance RAG, and with LangGraph, implementing these ideas becomes much simpler. So, if you’re interested in implementing these concepts, LangGraph is the tool for you.