LANGCHAIN — Can OpenGPTs Benefit from the Addition of Long-Term Memory?
The computer was born to solve problems that did not exist before. — Bill Gates
Long-term memory is a crucial aspect of conversational agents like OpenGPTs, allowing them to retain information over extended periods. Despite this, OpenGPTs and similar models mostly support basic conversational memory. In this article, we will discuss the concept of long-term memory, explore different types of memory, and walk through the specific long-term memory implementation in OpenGPTs through the creation of a “Dungeons and Dragons” Dungeon Master chatbot.
LLMs are Stateless
Long-Short Term Memory (LLMs) are stateless by nature, meaning that they do not remember previous inputs when operating on subsequent inputs. In turn, most LLMs are also stateless, necessitating the user to maintain the state before passing it to the LLM.
The first main exception was introduced by OpenAI’s Assistants API, allowing the tracking of a list of messages. This shift enables the API to maintain state, albeit the LLM itself remains stateless.
Conversation Memory
Conversational memory involves remembering previous messages in a conversation. This is implemented by tracking a list of previous messages and then passing them into the prompt, which in turn is fed into the LLM.
However, issues arise when the list of previous messages becomes excessively long, potentially exceeding the context window or becoming too distracting for the LLM to process effectively.
Semantic Memory
Semantic memory involves finding messages similar to the current message and incorporating them into the prompt. This is achieved by computing an embedding for each message and then identifying other messages with similar embeddings.
Yet, this approach has its drawbacks, including challenges in retrieving the right messages when insights or information are spread across multiple messages and not accounting for changes in preferences or information over time.
Generative Agents
Generative agents, introduced in a recent paper, tackle some of the challenges associated with memory. This approach utilizes recency, relevancy, and reflection to build memory using various techniques.
Long Term Memory Abstraction
The most general abstraction of long-term memory involves tracking a state that is updated at intervals and combined into the prompt. Implementing long-term memory involves addressing three key questions:
- What state is being tracked?
- How is the state updated?
- How is the state used?
This abstraction forms the basis for different memory types, such as conversation memory, semantic memory, and generative agents.
Application-Specific Memory
While generic memory forms are suitable for building AGI, application-specific memory can significantly enhance the reliability and performance of narrow applications. Defining the state being tracked, understanding how it is updated, and how it is used are essential considerations when building application-specific memory.
Dungeons and Dragons Chatbot
To showcase the implementation of long-term memory in OpenGPTs, we developed a chatbot designed to serve as a dungeon master for a game of Dungeons and Dragons. This involved tracking and updating two distinct states — character and quest.
Conclusion
Long-term memory is a relatively underexplored topic, due to its nature as either being very general and trending towards AGI, or highly application-specific. At LangChain, we emphasize the importance of application-specific memory and provide tools such as LangChain, OpenGPTs, and LangSmith to facilitate its implementation.
By applying the principles of long-term memory, we showcased a custom implementation through the creation of a “Dungeons and Dragons” chatbot using OpenGPTs. We believe that most applications requiring long-term memory benefit from application-specific memory, and we are committed to assisting companies in this regard.
In conclusion, long-term memory is a critical aspect of conversational agents, and its implementation should be tailored to the specific requirements of the application at hand.