LANGCHAIN — Foundations of LLM App Development with LangChain JS and Zep
Digital design is like painting, except the paint never dries. — Neville Brody
In this article, we will explore the foundations of LLM app development using LangChain.js and Zep. We will cover the creation of three foundational LLM apps, including a simple conversational bot, a Retrieval Augmented Generation (RAG) app, and a REACT-type agent. Each example will demonstrate the usage of LangChain’s ZepMemory and ZepVectorStore classes.
Building a Simple Conversational Bot
To begin, let’s initialize Zep in our app and create a session ID to represent the user or a chat session. Then, we’ll load some test data into the chat history for this session using the following code:
const client = await ZepClient.init(ZEP_API_URL, ZEP_API_KEY);
const sessionId = randomUUID();
const messages = history.map(({ role, content }) => new Message({ role, content }));
const zepMemory = new Memory({ messages });
await client.memory.addMemory(sessionId, zepMemory);Next, we’ll create an instance of ZepMemory and a conversation chain to recall the chat history. We'll ask the LLM what has been discussed so far, as shown in the following code snippet:
const model = new ChatOpenAI({ modelName: "gpt-3.5-turbo", temperature: 0 });
const memorySimple = new ZepMemory({ sessionId, baseURL: ZEP_API_URL, apiKey: ZEP_API_KEY });
const conversationChain = new ConversationChain({ llm: model, memory: memorySimple });
const res1 = await conversationChain.run("What have we discussed so far?");
console.log(res1);Building a Q&A over Docs / RAG-type App
In this example, we’ll use Zep’s ZepVectorStore to support a ConversationalRetrievalQAChain that searches over a Zep document collection. We'll load documents into the vector store and wait for Zep to finish embedding them, as shown in the following code snippet:
const vectorStore = await loadDocsIntoVectorStore(ZEP_COLLECTION_CONFIG);
// Wait for the ZepVectorStore to finish embedding the documents
while (true) {
const c = await client.document.getCollection(ZEP_COLLECTION_NAME);
console.log(`Embedding status: ${c.document_embedded_count}/${c.document_count} documents embedded`);
await new Promise((resolve) => setTimeout(resolve, 1000));
if (c.status === "ready") {
break;
}
}We’ll then configure the ZepMemory and ConversationalRetrievalQAChain classes and question the LLM about the books. The following code demonstrates the setup:
const memory = new ZepMemory({ sessionId, baseURL: ZEP_API_URL, apiKey: ZEP_API_KEY, memoryKey: "chat_history", inputKey: "question", outputKey: "text" });
const chain = ConversationalRetrievalQAChain.fromLLM(model, vectorStore.asRetriever({ searchType: "mmr", k: 4 }), { memory: memory });Building a REACT-type Agent with Zep Memory Retrieval and Search as Tools
Lastly, we’ll build an agent that uses Zep’s conversational history and vector store as tools. We’ll use the ZepRetriever for searching over the session's chat history and create tools for searching for authors, characters, and passages in sci-fi books. Below is an example of setting up the tools and initializing the agent:
const zepMemoryRetriever = await new ZepRetriever({ url: ZEP_API_URL, apiKey: ZEP_API_KEY, sessionId: sessionId });
const tools = [
new DynamicTool({
name: "peopleRetriever",
description: `call this if you want to search for authors, characters, or people we may have discussed in the past. input should be a search string`,
func: async (query) => await getPeopleFromMemoryTool(query, zepMemoryRetriever),
}),
new DynamicTool({
name: "bookSearch",
description: "call this if to search for passages in sci-fi books. input should be a search string",
func: async (query) => await getBookSearchTool(query, vectorStore),
}),
];
const executor = await initializeAgentExecutorWithOptions(tools, model, { agentType: "zero-shot-react-description" });These examples illustrate how to build foundational LLM apps using LangChain.js and Zep. They provide a solid starting point for building more advanced applications with LangChain’s features and Zep’s capabilities.
