Generative AI - Learn the LangChain Basics by Building a Berlin Travel Guide

LangChain is a framework that’s like a Swiss army knife for large language models (LLMs).

We’ll delve into LangChain’s components by implementing an engaging tour guide, making it an “all-knowing” Berlin expert. Follow along to see why LangChain is truly a game-changer by helping us easily create advanced LLM applications.

So fasten your seatbelts and prepare for an engaging journey. With our AI companion powered by LangChain and Large Language Models, we’ll immerse ourselves in Berlin’s culture. Get ready to experience the transformative potential of Large Language Models!”

Jump Directly to the Notebook

All the code for this article is ready to use in a Google Colab notebook. If you have questions, please reach out to me on LinkedIn.

Why do we need LangChain?

LangChain is a framework that offers various components to reduce the implementation time needed to build applications with Large Language Models. We will cover the most commonly used components in this article.

Key Components of LangChain

Models

Obviously, the most important component is the model.

Langchain provides a great abstraction that helps us switch the underlying LLM with just “one line of code”.

chat = ChatVertexAI()
chat = ChatOpenAI()

Interacting with different models like GPT-4 or PaLM 2 and other open-source models is getting much easier this way. No need to change the rest of the implementation. It’s a standardized interface for interacting with many different LLM providers and models. Considering the early stage of LangChain, this works indeed already very well, but there are some cases where you can feel the deep integration with OpenAI.

LangChain differentiates between two different model types.

Large Language Models Text completion models with simple input-output.

from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(temperature=0.0)
llm.predict("Tell me about the famous Berghain in Berlin")

Chat Model They are based on LLMs but trained to have conversations in addition to slightly different API usage.

from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    AIMessage,
    HumanMessage,
    SystemMessage
)

chat = ChatOpenAI(temperature=0)
chat.predict_messages([HumanMessage(
  content="Which museum can you recommend and how do I get there?")])

I generally prefer the Chat Models because they usually provide better results due to the structured prompt interface (AIMessage, SystemMessage, and HumanMessage). Therefore for the rest of this article, we focus on Chat Models. Side note: This structure also helps reduce prompt injection attacks because we can separate the user-provided context.

Memory

By nature, Large Language Models are stateless. They do not remember any history of our conversation. For instance, if you introduce yourself to the model and inquire about your name later, it won’t remember it by default.

This is a key distinguishing factor when compared to ChatGPT or Bard, which come integrated with memory.

If we use the LLMs, we need to take care of that ourselves, and for that LangChain provides memory.

When using LLMs, it becomes our responsibility to account for this memory, which is where LangChain steps in. LangChain introduces a memory component that records the message history. Both human input and llm output. This provides the model with context, allowing it to recall previous interactions.

Remember, implementing memory is crucial if you intend to build a chatbot with an LLM. However, if your goal is merely to classify content, the memory component is not required.”

memory = ConversationBufferMemory()
memory.chat_memory.add_user_message("Hi my name is Sascha")
memory.chat_memory.add_ai_message("Whats up Sascha?")
print(memory.buffer)

# Human: Hi my name is Sascha
# AI: Whats up Sascha?

There are multiple types of memory and integrations into databases, with BufferMemory as the most basic one. The other ones focus on reducing the context size by either keeping a maximum number of messages or even more intelligent a summarization. Which one to choose depends on the context size your model allows and the type of application you are building.

I can highly recommend vector memory. It stores all messages in a Vector Database and queries the top matching messages based on the user's input.

If you build a bot, you need to store the memory together with a user or session ID to retrieve it later if the user uses the chat again.

In the example above, we manually add the user and AI message to the LangChain Memory. There is an easier way to integrate this, but we first need to understand the concept of Chains in LangChain and then combine it with the memory.

Chains

The Chain interface allows us to combine the model with other components. Or we even can combine multiple chains with each other.

To connect our memory to our model, we need to chain them. The chain combines the llm and the memory into a chained application.

chat = ChatOpenAI(temperature=0)

memory = ConversationBufferMemory()

conversation = ConversationChain(
    llm=chat, 
    memory=memory
)

Chains are essential for every LLM application. There are multiple chain types, each solving a different use case. I highly recommend the LangChain documentation, as they have examples for each chain type.

• ConversationChain The standard chain chat model simply wants to combine it with a memory and a prompt.
• SimpleSequentialChain Connect multiple chains with each other in sequence.
• ConversationalRetrievalChain Combines a conversation chain with document retrieval to answer questions based on documents in a conversational way.
• SummarizationChain To summarize documents.
• APIChain If you want your application to use another API to retrieve information.
• RouterChain You can provide multiple chains, each with its own description. Based on the description, LLM will decide which chain to use. This is similar to the Agent approach, which we will cover later in the article.

Prompts

Prompts play a fundamental role in any Large Language Model. They act as the primary means of communication with the model, shaping the type and structure of the output generated. With LLMs, it’s all about prompts. Prompts can be simple but can also get very complex.

With Prompt Templates, we can build prompts that are capable of handling dynamic inputs.

from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate

chat = ChatOpenAI(temperature=0)

template = PromptTemplate.from_template("""
You are a experience tour guide in {city}. 
Provide the top 2 attraction worth to visit""")


prompt = template.format(city="Berlin")
output = chat.predict(prompt)
print(output)

prompt = template.format(city="New York")
output = chat.predict(prompt)
print(output)

There are multiple types of Prompt Templates:

• PromptTemplates The easiest variant is simply taking our prompt and parameterizing it by putting a placeholder into it. This way, we can dynamically change the value of our placeholders {city} using format.
• ChatPromptTemplate It allows us to define system messages which had be shown to be more effective in instructing the model. I mentioned that at the beginning of the article.
• FewShotPromptTemplate With few shot prompts, we provide the model with additional help to better understand how it should behave.
• CustomPromptTemplates That can be implemented according to your needs.

For the rest of the article, we use ChatPromptTemplate. The usage is actually not too complicated. Instead of a single prompt, we split out prompt into instructions for our model SystemMessagePromptTemplate and the human input HumanMessagePromptTemplate . With this approach, we have a clear separation.

from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain.prompts import PromptTemplate

from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
    MessagesPlaceholder
)

system_prompt = SystemMessagePromptTemplate.from_template("""
Act as a experienced berlin tour guide. 
Answer friendly but if you get a question that is not related to Berlin deny the answer with: "Ick mach dir jleich Beene!".
""")
memory_prompt = MessagesPlaceholder(variable_name="history")
human_prompt = HumanMessagePromptTemplate.from_template("{input}")

chat_prompt = ChatPromptTemplate.from_messages([
    system_prompt,
    memory_prompt,
    human_prompt])

memory = ConversationBufferMemory(return_messages=True,
        memory_key="history",)

conversation = ConversationChain(
    llm=llm,
    memory=memory,
    prompt=chat_prompt
)

As you see, we now combined our model with memory and our prompt. That’s already a full working bot. But we don’t stop here. There is more.

Indexes

Imagine we want to ask our AI Berlin Tourguide about the opening hours of a specific museum or the program for one of the famous techno clubs. It will not be capable of answering this question as this is not part of the model's training data.

The first simple approach we could add this information as part of our prompt. But we need to keep in mind the LLM token limits. All those information would be too large to put as context into our prompt. PaLM 2 has a token limit of 8196 which is approximately 5000 words (depending on the language).

That's why we use an index to retrieve only relevant information.

It is important to know that the information is stored as an embedding. This is needed because we encode our question and our information in the same embedding space.

Indexes come with a whole bunch of components for loading, splitting, and retrieving documents that are stored in a vector store.

from langchain.document_loaders import UnstructuredURLLoader
from langchain.document_loaders import TextLoader
from langchain.embeddings import VertexAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.text_splitter import CharacterTextSplitter

sites = [
    "https://www.smb.museum/en/museums-institutions/bode-museum/exhibitions/current/",
    "https://www.smb.museum/en/museums-institutions/bode-museum/plan-your-visit/prices-tickets/",
    "https://tresorberlin.com/events",
    "https://www.berghain.berlin/de/program",
    "https://www.katerblau.de"
]

loader = UnstructuredURLLoader(urls=sites)
documents = loader.load()

text_splitter = CharacterTextSplitter(chunk_size=1000)
documents = text_splitter.split_documents(documents)
print(documents)
print(len(documents))

embeddings = VertexAIEmbeddings()
vectorstore = Chroma.from_documents(documents, embeddings)

For example, the user might be interested in Ukrainian art. The index will retrieve the most relevant information based on the user's question and use this data to answer the question.

The rest of the documents will not be returned. This way, we can keep the context small and within the token limits.

relevat_information=vectorstore.similarity_search("I am interested in Ukrainian Art", k=5)
relevat_information
context = "\n".join([doc.page_content for doc in relevat_information])
context

To combine this with our model and memory, we use another a ConversationalRetrievalChain. If we now ask for the price for the famous Bode museum or a club lineup on a specific date, the LLM can use the retrieved documents to answer that question.

from langchain.chains import ConversationalRetrievalChain

template = """
Act as a experienced berlin tour guide. 
Answer friendly but if you get a question that is not related to Berlin deny the answer with: "Ick mach dir jleich Beene!".
Only answer questions within the <tag> section
Today is Friday the 14th of July 2023

context:
{context}

chat history:
{chat_history}

tourist question:
<tag>{question}<tag>

you as tour guide need to obey rules when answering:
1. only answer questions about berlin. if there is another question answer with "Can I help you with something else about Berlin?"
"""


PROMPT = PromptTemplate(
    input_variables=["chat_history", "question", "context"], 
    template=template
)

memory = ConversationBufferMemory(
    memory_key='chat_history', 
    return_messages=True, 
    output_key='answer'
)

qa = ConversationalRetrievalChain.from_llm(
    llm=ChatVertexAI(temperature=0.5),
    retriever=vectorstore.as_retriever(k=1),
    memory=memory,
    return_source_documents=True,
    combine_docs_chain_kwargs={"prompt": PROMPT}
)

We kept our AI Berlin Tourguide simple by keeping the documents in a memory index, but if you are interested in a production scale approach, check out my other article about Document Retrieval and Question Answering.

Agents

Agents are one of the most interesting components of LangChain. With agents, we are even more flexible. The agents can decide on their own which and how they solve a problem based on a set of available tools.

I cover LangChain Agents in a dedicated article. Follow me if you don’t want to miss that.

Generative AI Series

I’ve written a series of articles, and there’s more to come. Stay tuned by following me.

1. Generative AI — The Evolution of Machine Learning Engineering
2. Generative AI — Getting Started with PaLM 2
3. Generative AI — Best Practices for LLM Prompt Engineering
4. Generative AI — Document Retrieval and Question Answering with LLMs
5. Generative AI — Mastering the Language Model Parameters for Better Outputs
6. Generative AI — Understand and Mitigate Hallucinations in LLMs
7. Generative AI — Learn the LangChain Basics by Building a Berlin Travel Guide
8. Generative AI — Image Generation using Vertex AI Imagen
9. Generative AI — Protect your LLM against Prompt Injection in Production
10. Generative AI — AWS Bedrock Approach vs. Google & OpenAI
11. Generative AI — How to Fine Tune LLMs
12. more to come over the next weeks

Thanks for reading

Your feedback and questions are highly appreciated. You can find me on LinkedIn or connect with me via Twitter @HeyerSascha. Even better, subscribe to my YouTube channel ❤️.