LANGCHAIN — LangChain Partners with CommandBar on Their CoPilot User Assistant

The most dangerous phrase in the language is, ‘We’ve always done it this way.’ — Grace Hopper.

LangChain has recently partnered with CommandBar on their CoPilot User Assistant. This collaboration has led to the integration of LangSmith, which has significantly improved CommandBar’s user assistance capabilities. In this tutorial, we will explore how LangSmith was used by CommandBar to enhance their CoPilot’s performance and deliver better experiences for their customers. We will delve into the four main ways in which LangSmith aided the CommandBar team: Trace Visibility, Debugging, Increased testing coverage, and Monitoring.

Trace Visibility: LangSmith allowed the team to monitor user interactions with the Copilot by setting up traces. This visibility into the conversation threads enabled CommandBar to proactively identify and address any potential issues or bad experiences that users may encounter. The following code snippet demonstrates how traces were set up using LangSmith decorators:

from langsmith import trace

@trace
def user_interaction_handler(user_input):
    # code to handle user interaction
    pass

Debugging: LangSmith traces provided valuable insights into the retrieval system of the Copilot. By analyzing the traces, the team could determine if the correct documents were being retrieved and take necessary measures to improve the system. Here’s an example of how LangSmith traces were used for debugging:

from langsmith import trace

@trace
def retrieval_system_handler(query):
    # code for document retrieval
    pass

Increased testing coverage: The adoption of LangSmith resulted in a significant increase in the number of tests that could be performed on new code changes. This automated testing approach supplemented manual QA and improved the evaluation of the Copilot’s responses. The code snippet below illustrates how LangSmith facilitated automated testing:

from langsmith import test

@test
def test_new_code_change():
    # code for testing new feature
    pass

Monitoring: LangSmith was utilized to monitor the overall health of the application and to alert the CommandBar team in case of any issues or outages with the LLM provider. The monitoring tab in LangSmith provided a comprehensive view of the application’s performance. The following code snippet demonstrates how LangSmith was used for monitoring:

from langsmith import monitor

@monitor
def monitor_application_health():
    # code for monitoring application health
    pass

In conclusion, the integration of LangSmith has empowered CommandBar to enhance the performance of their CoPilot User Assistant, resulting in better user experiences and a decrease in support tickets. By leveraging LangSmith for trace visibility, debugging, increased testing coverage, and monitoring, CommandBar has been able to proactively address user needs and continuously improve the Copilot’s capabilities. This tutorial has provided insights into how LangSmith can be utilized to streamline user assistance and improve the overall performance of AI-powered applications.