Function calling in Ollama 0.3.0
Tool use with any model privately with LLama3.1
Function calling is bridging the gap between language models and real-world applications. It allows an AI system to seamlessly interact with external tools (for example API-s, databases, calculators), dramatically expanding its problem-solving abilities. In this article we will explore the newly introduced function calling in Ollama framework, give an example of defining two function environments, and how it gracefully coexists with traditional conversational AI workflows.
Requirements: ollama>=0.3.0 llama3.1 and compatible models
Code walk-through
The following code block is an example of using ollama (0.3.0) with the `tools` block in the ollama.chat object. Here we are defining two function, one for the weather, and one for number comparison. This list of functions can be arbitrarily numerous.
import ollama
import requests
def get_current_weather(city):
base_url = f"https://wttr.in/{city}?format=j1"
response = requests.get(base_url)
data = response.json()
return f"Temp in {city}: {data['current_condition'][0]['temp_C']}"
def what_is_bigger(n, m):
if n > m:
return f"{n} is bigger"
elif m > n:
return f"{m} is bigger"
else:
return f"{n} and {m} are equal"
def chat_with_ollama_no_functions(user_question):
response = ollama.chat(
model='llama3.1:8b-instruct-fp16',
messages=[
{'role': 'user', 'content': user_question}
]
)
return response
def chat_with_ollama(user_question):
response = ollama.chat(
model='llama3.1:8b-instruct-fp16',
messages=[
{'role': 'user', 'content': user_question}
],
tools=[
{
'type': 'function',
'function': {
'name': "get_current_weather",
'description': "Get the current weather for a city",
'parameters': {
'type': "object",
'properties': {
'city': {
'type': "string",
"description": "City",
},
},
'required': ['city'],
},
},
},
{
'type': "function",
'function': {
"name": "which_is_bigger",
'parameters': {
'type': 'object',
'properties': {
'n': {
'type': "float",
},
"m": {
'type': "float"
},
},
'required': ['n', 'm'],
},
},
},
],
)
return response
def main():
while True:
user_input = input("Enter your question (or 'quit' to exit): ")
if user_input.lower() == 'quit':
break
response = chat_with_ollama(user_input)
if 'tool_calls' in response['message'] and response['message']['tool_calls']:
tools_calls = response['message']['tool_calls']
for tool_call in tools_calls:
tool_name = tool_call['function']['name']
arguments = tool_call['function']['arguments']
if tool_name == 'get_current_weather' and 'city' in arguments:
result = get_current_weather(arguments['city'])
print("Weather function result:", result)
elif tool_name == 'which_is_bigger' and 'n' in arguments and 'm' in arguments:
n, m = float(arguments['n']), float(arguments['m'])
result = what_is_bigger(n, m)
print("Comparison function result:", result)
else:
print(f"No valid arguments found for function: {tool_name}")
else:
# If no tool calls or no valid arguments, use the LLM's response
response = chat_with_ollama_no_functions(user_input)
print("AI response:", response['message']['content'])
if __name__ == "__main__":
main()The most important part here is the definition of the `tools` argument with its parameters, i.e. the extraction of the relevant values from the user prompt and defining them as arguments to the functions (`get_current_weather`, `what_is_bigger`, respectively).
The main function just handles the chat and channels the extracted arguments to the corresponding functions. Also, in case that during the chat flow, there are no keywords (arguments for the functions) have been detected, the chat uses the default response of the LLM.
The “tools” field in detail
The `chat_with_ollama` function is a crucial component of this code, responsible for interacting with the Ollama language model and setting up the function-calling capabilities.
def chat_with_ollama(user_question):
response = ollama.chat(
model='llama3.1:8b-instruct-fp16',
messages=[
{'role': 'user', 'content': user_question}
],
tools=[
# Tool definitions here
]
)
return responseOllama Chat Configuration The function uses `ollama.chat()` to interact with the Ollama API. Next to the model and messages sections, it now has the `tools` section.
Tools Section The `tools` array is where the function-calling capabilities are defined. Each tool is an object with the following structure:
{
'type': 'function',
'function': {
'name': "function_name",
'description': "Function description",
'parameters': {
'type': "object",
'properties': {
# Parameter definitions
},
'required': ['param1', 'param2'],
},
},
}Two tools are defined in this function: `get_current_weather` with only the `city` as argument, and the `which_is_bigger`with the floats as parameters.
These tool definitions inform the Ollama model about the available functions it can call. When the model determines that a function call is necessary to answer a user’s question, it will include a `tool_calls` object in its response, specifying which function to call and with what arguments.
The main program then interprets these `tool_calls` and executes the appropriate functions, integrating their results into the conversation flow.
Function calling in general
In this implementation, the model can recognize when a user’s query requires specific data or computations that it can’t directly provide. Instead of guessing or hallucinating, it can “call” predefined functions to retrieve accurate information or perform calculations. This is achieved by providing the model with a list of available functions and their parameters. When the model determines a function call is necessary, it generates a structured request specifying the function name and required arguments, allowing the application to execute the appropriate function and incorporate the result into the conversation.
The chat_with_ollama function defines these tools, each with its own set of parameters and descriptions. During execution, the application checks for tool_calls in the model's response. If present, it iterates through each call, matches it to the corresponding function, and executes it with the provided arguments. By simply adding new functions to the tools list and implementing their corresponding logic in the main loop, the capabilities of the app will be more specific to the needs of the user.
Fallback to default LLM response
The case when a chat context does not execute functions: The code also handles scenarios where function calls are not necessary or applicable. In cases where the model’s response doesn’t include tool_calls, or when the arguments for a function call are invalid, the application falls back to a standard chat interaction. This is implemented through the chat_with_ollama_no_functions function, which processes the user's input without any function-calling capabilities. This fallback ensures that the conversation can continue smoothly even when the user's query doesn't require external data or computations, maintaining a natural dialogue flow. It's a crucial feature that allows the AI to handle a wide range of queries, from those requiring specific data to open-ended conversations.
Conclusion
This code example shows a simple implementation of multiple functions in a chat application using ollama. While running simple chats with the code, one can see that when the pattern is defined in a user prompt, the corresponding functions are executed as expected and return a response as defined in the function. When no tool calling patterns were detected, the app uses the LLM’s response (no function calling). In this simplistic example we used only two functions. This can be more complex, in case when several similar functions are defined, with similar arguments. This behavior of such a case should also be tested.
In Plain English 🚀
Thank you for being a part of the In Plain English community! Before you go:
- Be sure to clap and follow the writer ️👏️️
- Follow us: X | LinkedIn | YouTube | Discord | Newsletter
- Visit our other platforms: CoFeed | Differ
- More content at PlainEnglish.io
