LangChain 🦜🔗: Using different LangChain chains to write a new episode for The Office US

In this article, I will show you how to:

• Create a chat model
• Create a prompt template
• Use several chains in LangChain like Sequential Chains, Summarisation, Question Answering and Bash chains

LangChain is the next big chapter in the AI revolution.

In my last article, I explained what LangChain is and how to create a simple AI chatbot that can answer questions using OpenAI’s GPT language model and give GPT internet access. If you haven't read it, be sure to check it out:

Setting up LangChain

The following code can be found in my GitHub repository.

To start off with, we need to install and import all the necessary packages to use LangChain in your IDE:

# install the packages
!pip install -Uqqq pip --progress-bar off
!pip install -qqq langchain==0.0.149 --progress-bar off
!pip install -qqq openai==0.27.4 --progress-bar off
!pip install -qqq tiktoken==0.3.3 --progress-bar off
!pip install -qqq watermark==2.3.1 --progress-bar off
!pip install -qqq chromadb==0.3.21 --progress-bar off

# import packages
import os
import textwrap
from getpass import getpass
import chromadb
import langchain
import openai

from langchain.chains import (
LLMBashChain,
LLMChain,
RetrievalQA,
SimpleSequentialChain
)
from langchain.chains.summarize import load_summarize_chain
from langchain.chat_models import ChatOpenAI
from langchain.docstore.document import Document
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma

import credentials

Let us create a chat model with a prompt template. I defined a function that wraps the response of the models. This function aims to print a formatted response text in a visually appealing way, ensuring that each line of the response does not exceed a certain width.

def print_response(response: str):
    print("\n".join(textwrap.wrap(response, width=100)))

Create an API key, in this case, I will use the OpenAI GPT large language model. If you do not have an OpenAI API key, you can create one here.

I stored my API key in another file called credentials.

OPENAI_API_KEY = getpass()
os.environ["OPENAI_API_KEY"] = credentials.OPENAI_API_KEY

Now, let's create the model and the prompt template.

model = ChatOpenAI(temperature=0.3, model_name="gpt-3.5-turbo")

This model has a temperature of 0.3. This corresponds with the randomness of the output of the model. The specific GPT model that we will be using is the gpt-3.5-turbo model.

As I am a huge The Office fan and I can’t get enough of it (that's what she said), I will create new ideas and scripts for a new episode of the office in my examples.

template = """
You have to come up with location to shoot (along with a 20-50 word description)
for a new episode of the TV show "The Office" based on the theme.

{theme_suggestion}

ANSWER:
"""
prompt = PromptTemplate(input_variables=["theme_suggestion"], template=template)

location_chain = LLMChain(llm=model, prompt=prompt, verbose=True)

After defining the prompt template, we can start testing the model that we created by giving an import for theme suggestion. Let's test the model for the input “Visiting Europe”

response = location_chain("Visiting Europe")
response

print_response(response["text"])

The output speaks for itself I think:

“The Office crew heads to Paris, France for a company retreat. While there, they struggle with language barriers, cultural differences, and a surprise visit from a former employee. Will they be able to bond and come together as a team in the City of Love?”

This is a beautiful idea for a new episode, it worked!

Sequential chains

The next step after calling a language model is to make a series of calls to a language model. This is particularly useful when you want to take the output from one call and use it as the input to another.

In this article we will walk through some examples of how to do this, using sequential chains.

Sequential chains are defined as a series of chains, called in deterministic order. There are two types of sequential chains:

SimpleSequentialChain: The simplest form of sequential chains, where each step has a singular input/output, and the output of one step is the input to the next.

SequentialChain: A more general form of sequential chains, allowing for multiple inputs/outputs.

In this example, we will use a similar prompt template as before:

template = """
Generate a short dialogue between Jim and Pam
from the TV show "The Office" for a new episode based on the location

{location}

ANSWER:
"""
prompt = PromptTemplate(input_variables=["location"], template=template)

conversation_chain = LLMChain(llm=model, prompt=prompt, verbose=True)

Let's initiate a SimpleSequentialChain.

sequential_chain = SimpleSequentialChain(
    chains=[location_chain, conversation_chain], verbose=True
)
response = sequential_chain.run("Visiting Europe")

#%%
conversation = response
print(conversation)

The script:

Jim: “I can’t believe we got lost in Paris, of all places.”

Pam: “I know, right? But I have to admit, it’s kind of romantic wandering around these streets with you.”

Jim: “Yeah, it’s like we’re in our own little movie.”

Pam: “Speaking of movies, we should find a cute little cafe and have some croissants and coffee.”

Jim: “Sounds perfect. But first, let’s take a selfie in front of the Eiffel Tower.”

Pam: “Yes! And then we can send it to Dwight and Michael to make them jealous.”

Jim: “Ha! They’re probably struggling to order food in French right now.”

Pam: “Well, at least we have each other to navigate this city with.”

Jim: “Always, Pam. Always.”

Summarisation

A big application for Large language models is to get a summary from a big piece of text or document. There are 4 different chain types that can help you with this, all with their own pros and cons.

Stuffing The simplest approach is to “stuff” all relevant information into the prompt as context to be passed to the language model. The StuffDocumentsChain in LangChain implements this.

Pros: Only makes a single call to the LLM. When generating text, the LLM has access to all the data at once.

Cons: Most LLMs have a context length, and for large documents (or many documents) this will not work as it will result in a prompt larger than the context length.

The main downside of this method is that it only works on smaller pieces of data. Once you are working with many pieces of data, this approach is no longer feasible. The next two approaches are designed to help deal with that.

Map Reduce This method involves running an initial prompt on each chunk of data (for summarisation tasks, this could be a summary of that chunk; for question-answering tasks, it could be an answer based solely on that chunk). Then a different prompt is run to combine all the initial outputs. This is implemented in the LangChain as the MapReduceDocumentsChain.

Pros: Can scale to larger documents (and more documents) than StuffDocumentsChain. The calls to the LLM on individual documents are independent and can therefore be parallelised.

Cons: Requires many more calls to the LLM than StuffDocumentsChain. Loses some information during the final combined call.

Refine This method involves running an initial prompt on the first chunk of data, generating some output. For the remaining documents, that output is passed in, along with the next document, asking the LLM to refine the output based on the new document.

Pros: Can pull in more relevant context, and may be less lossy than MapReduceDocumentsChain.

Cons: Requires many more calls to the LLM than StuffDocumentsChain. The calls are also NOT independent, meaning they cannot be paralleled like MapReduceDocumentsChain. There is also some potential dependencies on the ordering of the documents.

Map-Rerank (not implemented for summarisation) This method involves running an initial prompt on each chunk of data, that not only tries to complete a task but also gives a score for how certain it is in its answer. The responses are then ranked according to this score, and the highest score is returned.

Pros: Similar pros as MapReduceDocumentsChain. Requires fewer calls, compared to MapReduceDocumentsChain.

Cons: Cannot combine information between documents. This means it is most useful when you expect there to be a single simple answer in a single document.

Source: LangChain documentation.

Let's do an example using a Stuffing chain:

template = """
Write a concise bullet list summary of the conversation between Jim and Pam from the TV show "The Office":

{text}

Concise summary using markdown:"""

prompt = PromptTemplate(template=template, input_variables=["text"])
summary_chain = load_summarize_chain(
    model, chain_type="stuff", verbose=True, prompt=prompt
)

docs = [Document(page_content=conversation)]
docs

summary_result = summary_chain.run(docs)
print(summary_result)

Summary output:

• Jim and Pam got lost in Paris.
• They find it romantic to wander around the streets together.
• They plan to find a cute cafe and have croissants and coffee.
• They want to take a selfie in front of the Eiffel Tower and send it to Dwight and Michael to make them jealous.
• They are grateful to have each other to navigate the city with.

Question Answering

Here we look at how to use LangChain for question-answering over a list of documents. The same four chain types that have been listed before can be used.

template = """
You have to come up with a 200-300 word script for a new episode
of the TV show "The Office" based on the theme

{theme_suggestion}

ANSWER:
"""
prompt = PromptTemplate(input_variables=["theme_suggestion"], template=template)

script_chain = LLMChain(llm=model, prompt=prompt, verbose=True)

script_response = script_chain("Going to the moon")

script = script_response["text"]
print(script)

I will give the output in the appendix of this article to not take up too much space. Give it a read, it is definitely worth it!

Now, let's split up the script into chunks of size 2048 characters with an overlap of 32 characters. The code’s objective is to divide a lengthy text file into manageable text portions for additional processing or analysis. For a variety of reasons, including enhancing computational performance, permitting parallel processing, or enabling sequential processing of smaller areas, breaking up huge texts into smaller pieces can be beneficial. When working with huge texts or implementing text-based models and algorithms that have restrictions on the input size, it enables more manageable and granular handling of text data.

script_docs = [Document(page_content=script)]

text_splitter = CharacterTextSplitter(chunk_size=2048, chunk_overlap=32)
texts = text_splitter.split_documents(script_docs)
len(texts)

The text was split up into 2 documents.

Now let us create a database with embeddings. Creating a database with embeddings in the field of Natural Language Processing (NLP) offers several advantages. Storing embeddings, which are numerical representations of text, allows for efficient similarity searches, enabling the retrieval of similar documents or sentences. It also speeds up computations by avoiding the need to recompute embeddings. The scalability of such a database accommodates large volumes of text data, making it suitable for diverse applications. Click this link for a deeper understanding of embeddings.

embeddings = OpenAIEmbeddings()

#%%
db = Chroma.from_documents(texts, embeddings)

Create a question-answering chain

qa_chain = RetrievalQA.from_chain_type(
    llm=model, chain_type="stuff", retriever=db.as_retriever(search_kwargs={"k": 2}, verbose=True)
)

response = qa_chain("What is the place that The Office team is visiting?")
response
print_response(response["result"])

Output: “The Office team is visiting the moon.”

Let's test if it can answer a question about something that is not in the text

response = qa_chain("Why the client doesn't want to deal with Michael?")
print_response(response["result"])

Output: “There is no information provided in the context that suggests the client doesn’t want to deal with Michael.”

This shows that our model does not makeup things about the text when certain information is not provided. That is very important!

Bash Chain

The last chain application I will shortly touch upon is BashChain. The BashChain is a special chain that allows you to run bash commands. It is useful for chaining together bash commands with other chains.

Here is an example of how it works:

text = """
Please write a bash script that prints a single line that Michael G. Scott from "The Office" might say
"""

bash_chain = LLMBashChain(llm=OpenAI(temperature=0), verbose=True)

bash_chain.run(text)

End note

There is a lot more you can do with LangChain and I would recommend playing around with it.

I hope this article helps to create a general understanding of LangChain chains and how to use LangChain in Python.

Illustration:

The wonderful illustration is made by Kateryna Malachkova. Be sure to check out her Medium and Instagram!

• https://medium.com/@malachkovak
• https://instagram.com/ph_malachkova?igshid=ZjE2NGZiNDQ=

Also, be sure to follow me on GitHub LinkedIn and if you like the work that I'm doing you could buy me a coffee:

• https://github.com/rubentak
• https://www.linkedin.com/in/ruben-tak-665b66194/
• https://www.buymeacoffee.com/rubentak

Appendix

Script for a The Office episode in which they go to the moon:

FADE IN:

INT. DUNDER MIFFLIN SCRANTON — DAY

The employees of Dunder Mifflin are gathered in the conference room for a meeting. Michael Scott is standing at the front of the room, holding a toy rocket ship.

MICHAEL: Good morning, everyone! Today, we’re going to talk about the moon.

JIM: (whispering to Pam) Is he serious?

PAM: (whispering back) I don’t know, but I’m afraid to ask.

MICHAEL: As you all know, NASA is planning a mission to the moon in a few years. And I’ve been thinking, why should they have all the fun?

DWIGHT: (excitedly) Are you suggesting we go to the moon, Michael?

MICHAEL: (nodding) Yes, Dwight. I am.

JIM: (sarcastically) Oh, great. Another one of Michael’s brilliant ideas.

MICHAEL: (ignoring Jim) I’ve already contacted a space travel agency and they’ve agreed to take us to the moon.

PAM: (concerned) Michael, I don’t think that’s a good idea. Going to the moon is dangerous.

MICHAEL: (defensive) Pam, I’m not going to let a little thing like danger stop us from achieving our dreams.

The employees exchange worried glances.

CUT TO:

INT. SPACE TRAVEL AGENCY — DAY

Michael, Dwight, Jim, and Pam are standing in front of a large rocket ship.

MICHAEL: (excitedly) This is it, guys. Our ticket to the moon.

JIM: (sarcastically) Yay.

PAM: (nervously) Michael, are you sure about this?

MICHAEL: (confidently) Absolutely, Pam. Trust me, this is going to be the adventure of a lifetime.

The employees reluctantly board the rocket ship.

CUT TO:

INT. ROCKET SHIP — DAY

The employees are strapped into their seats, looking nervous.

MICHAEL: (over the intercom) Attention, everyone. We are about to take off. Please fasten your seatbelts and prepare for liftoff.

The rocket ship shakes and rumbles as it takes off.

CUT TO:

INT. MOON — DAY

The employees are standing on the surface of the moon, wearing space suits.

JIM: (in awe) Wow. We’re actually on the moon.

DWIGHT: (excitedly) This is amazing. I can’t wait to explore.

PAM: (worriedly) Michael, we need to get back to Earth. We can’t stay here forever.

MICHAEL: (disappointed) Fine. Let’s go back.

The employees board the rocket ship and take off.

CUT TO:

INT. DUNDER MIFFLIN SCRANTON — DAY

The employees are back in the conference room, looking exhausted.

MICHAEL: (smiling) Well, that was quite an adventure, wasn’t it?

JIM: (sarcastically) Yeah, Michael. It was a blast.

PAM: (smiling) I have to admit, it was pretty cool.

DWIGHT: (excitedly) Can we go back to the moon someday?

MICHAEL: (smiling) Who knows, Dwight. Maybe we will.

FADE OUT.