LANGCHAIN — Are JSON-Based Agents Compatible with Ollama and LangChain?

Software is like entropy: It is difficult to grasp, weighs nothing, and obeys the Second Law of Thermodynamics; i.e., it always increases. — Norman Augustine.

JSON-Based Agents with Ollama and LangChain: A Tutorial

Implementing an open-source Mixtral agent that interacts with a graph database like Neo4j through a semantic layer can significantly enhance the capabilities of LLMs by providing them with additional tools. This tutorial will guide you through the implementation of a JSON-based LLM agent using Mixtral 8x7b and LangChain. We will explore the tools in the semantic layer, JSON-based prompts for LLM agents, defining tool inputs in the system prompt, and showcasing some code snippets for clarity.

Tools in the semantic layer

In the LangChain documentation, tools with a single string input are commonly used. However, more complex inputs are also supported. For example, a recommender tool may have two optional inputs, such as movie and genre.

all_genres = [
    "Action",
    "Adventure",
    "Animation",
    ...
    "Western",
]

class RecommenderInput(BaseModel):
    movie: Optional[str] = Field(description="movie used for recommendation")
    genre: Optional[str] = Field(
        description=(
            "genre used for recommendation. Available options are:" f"{all_genres}"
        )
    )

JSON-based prompt for an LLM agent

The JSON-based prompt instructs the LLM on how to produce a JSON when it wants to use any available tools. It defines the output structure for the LLM, guiding it on how to use tools and provide the action input as a JSON blob.

{
  "action": $TOOL_NAME,
  "action_input": $INPUT
}

The full output should have the following structure:

Thought: you should always think about what to do
Action:
$JSON_BLOB
Observation: the result of the action
... (this Thought/Action/Observation can repeat N times)
Final Answer: the final answer to the original input question

Defining tool inputs in the system prompt

LangChain provides a function that transforms the custom Pydantic tool input definition into a JSON object that Mixtral recognizes. This allows you to define tools with their descriptions and input parameters.

from langchain.tools.render import render_text_description_and_args

tools = [RecommenderTool(), InformationTool(), Smalltalk()]
tool_input = render_text_description_and_args(tools)
print(tool_input)

Produces the following string description:

"Recommender":"useful for when you need to recommend a movie",
"args":{
   {
      "movie":{
         {
            "title":"Movie",
            "description":"movie used for recommendation",
            "type":"string"
         }
      },
      "genre":{
         {
            "title":"Genre",
            "description":"genre used for recommendation. Available options are:['Action', 'Adventure', 'Animation', 'Children', 'Comedy', 'Crime', 'Documentary', 'Drama', 'Fantasy', 'Film-Noir', 'Horror', 'IMAX', 'Musical', 'Mystery', 'Romance', 'Sci-Fi', 'Thriller', 'War', 'Western']",
            "type":"string"
         }
      }
   }
},
...

Conclusion

Implementing a JSON-based agent was made possible by LangChain and the work of the LangChain team. By utilizing the tools and prompts discussed in this tutorial, you can enhance the performance of LLMs like Mixtral as agents. The code for this implementation is available as a Langchain template and as a Jupyter notebook.

With this guide, you can now implement a JSON-based agent that interacts with services like Neo4j through a semantic layer using LangChain. It is a powerful technique that can significantly enhance the capabilities of language models by providing dynamic, real-time access to information and personalization through memory, resulting in a more interactive and adaptive user experience.