LANGCHAIN — What Is The Purpose of Building Chat Langchain 2?

It’s not that we use technology, we live technology. — Godfrey Reggio

Building Chat LangChain 2 serves the purpose of demonstrating the power of LangChain in simplifying the creation of Large Language Model (LLM) applications. In this tutorial, we will walk through the process of building a chatbot called Chat LangChain that answers questions about LangChain by indexing and searching through the Python docs and API reference. We will cover how to use the LangChain Expression Language (LCEL) to define a RAG chain, evaluate an LLM application, deploy a LangChain application, and monitor its performance.

Architecture

Ingestion

To perform RAG, we need to create an index over some source of information about LangChain that can be queried at runtime. Ingestion refers to the process of loading, transforming, and indexing the relevant data sources. The ingestion pipeline includes the following steps:

1. Load: We use SitemapLoader and RecursiveURLLoader to scrape the Python docs and API Reference. This approach is chosen for its ability to handle autogenerated content and its generalizability. Below is an example of loading the Python docs using SitemapLoader:

docs = SitemapLoader(
    "https://python.langchain.com/sitemap.xml",
    filter_urls=["https://python.langchain.com/"],
    parsing_function=langchain_docs_extractor,
    default_parser="lxml",
    bs_kwargs={
        "parse_only": SoupStrainer(
            name=("article", "title", "html", "lang", "content")
        ),
    },
    meta_function=metadata_extractor,
).load()

1. Transform: The RecursiveCharacterTextSplitter is used to partition the content into equally-sized chunks:

transformed_docs = RecursiveCharacterTextSplitter(
    chunk_size=4000, 
    chunk_overlap=200,
).split_documents(docs + api_ref)

1. Embed + Store: We use OpenAI’s embeddings to represent each chunk as a vector in the Weaviate vector store:

client = weaviate.Client(
    url=WEAVIATE_URL,
    auth_client_secret=weaviate.AuthApiKey(api_key=WEAVIATE_API_KEY),
)
embedding = OpenAIEmbeddings(chunk_size=200)
vectorstore = Weaviate(
    client=client,
    index_name=WEAVIATE_DOCS_INDEX_NAME,
    text_key="text",
    embedding=embedding,
    by_text=False,
    attributes=["source", "title"],
)

1. Indexing + Record Management: We use the LangChain Indexing API and a SQLRecordManager to track writes to the vector store and handle deduplication and cleanup of documents.

record_manager = SQLRecordManager(
    f"weaviate/{WEAVIATE_DOCS_INDEX_NAME}", db_url=RECORD_MANAGER_DB_URL
)
record_manager.create_schema()
indexing_stats = index(
    transformed_docs,
    record_manager,
    vectorstore,
    cleanup="full",
    source_id_key="source",
)

Continuous Ingestion

To keep the chatbot up-to-date with new LangChain releases and documentation, we add a scheduled Github Action to the repo that runs the ingestion pipeline daily.

Question-Answering

The question-answering chain involves using a prompt to rephrase the question, retrieving relevant documents, and then passing the context to a model to generate a response. Here’s an example of the chain:

condense_question_chain = (
    PromptTemplate.from_template(REPHRASE_TEMPLATE)
    | llm
    | StrOutputParser()
).with_config(
    run_name="CondenseQuestion",
)
retriever_chain = condense_question_chain | retriever

_context = RunnableMap(
    {
        "context": retriever_chain | format_docs,
        "question": itemgetter("question"),
        "chat_history": itemgetter("chat_history"),
    }
).with_config(run_name="RetrieveDocs")
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", RESPONSE_TEMPLATE),
        MessagesPlaceholder(variable_name="chat_history"),
        ("human", "{question}"),
    ]
)

response_synthesizer = (prompt | llm | StrOutputParser()).with_config(
    run_name="GenerateResponse",
)
answer_chain = _context | response_synthesizer

Evaluation

LangSmith is used to benchmark each version of the chatbot. The evaluation process involves interacting with the app, viewing the trace for bad examples, assigning blame to the different components, and making changes to the bot’s prompt, retriever, and overall architecture.