LANGCHAIN — What Are the Lessons from Deploying LLMs with Langsmith?

The function of good software is to make the complex appear to be simple. — Grady Booch

Deploying Language Model Management Systems (LLMS) with Langsmith can be a challenging yet rewarding process. In a recent webinar hosted by LangChain, Docugami shared their valuable learnings and insights on deploying LLMS with Langsmith. Let’s delve into the key lessons from the webinar and explore code examples to understand the practical implementation of LLMS with Langsmith.

Lesson 1: Real documents are more than flat text Docugami highlighted the structural complexity of real documents, such as Scanned PDFs, Digital PDFs, DOCX, and DOC. They emphasized the need to handle complex reading orders, including tables and multi-column flows. To address this, they discussed the use of LangChain’s expressive API and the LangSmith tool to handle real-world document structures effectively.

Lesson 2: Documents are Knowledge Graphs Docugami showcased their hierarchical Document XML Knowledge Graph, which contains deep hierarchies, custom semantic labels, and complex relationships expressed using the XML Data Model. They demonstrated how to leverage the knowledge graph for tasks such as Retrieval Augmented Generation (RAG) using code examples.

Lesson 3: Building Complex Chains with the LangChain Expression Language The webinar covered the creation of complex chains with LangChain’s Expression Language. This involved dealing with parallel branches, output parsers, conditional sub-chains, and more. A practical example of SQL generation with agent-like fixup for invalid SQL was presented, along with insights on navigating these complex chains using the LangSmith tool.

Lesson 4: Debugging Complex Chain Failures in Production Inevitably, issues arise when deploying LLMS in production. The webinar provided tips on debugging complex chain failures, including handling context length overflow and exceptions in output parsers. It emphasized the importance of making run traces in LangSmith more debuggable for effective troubleshooting.

Lesson 5: Docugami’s end to end LLM Ops with LangChain + LangSmith Lastly, Docugami summarized their overall flow for deploying models, monitoring real customer use, identifying problematic runs, and fixing these runs manually or with the help of other LLMS offline. This encompassed a holistic approach to LLM Ops and showcased the potential for further collaboration with LangSmith to enhance tooling.

To exemplify the practical implementation of these lessons, let’s consider a code snippet demonstrating the creation of a complex chain using LangChain’s Expression Language:

from langchain import Expression

# Define a complex chain with parallel branches and conditional sub-chains
complex_chain = (
    Expression("branch_1")
    .parallel(
        Expression("sub_chain_1").condition("condition_1"),
        Expression("sub_chain_2").condition("condition_2")
    )
    .outputParser("output_parser")
)

In conclusion, deploying LLMS with Langsmith involves addressing the intricacies of real-world documents, leveraging knowledge graphs, building complex chains, debugging production failures, and streamlining end-to-end LLM Ops. By incorporating the lessons shared in the webinar and utilizing LangChain’s capabilities, developers can effectively navigate the complexities of LLMS deployment.

Remember, the journey of deploying LLMS with Langsmith may pose challenges, but with the right tools and insights, you can simplify the complex and achieve remarkable results.