Teaching AI to Call APIs: Exploring Function Calling

In my previous article, I discussed the mechanics and benefits of using RAFT with Large Language Models (LLMs). Here, we dive deeper into the method of function calling — an essential capability that enables AI to interact and connect more efficiently by aptly leveraging API calls.

Introduction

As we see advancements in LLMs such as GPT-4 Turbo, GPT-4, and GPT-3.5 Turbo, Claude Opus, Gemini, the automated triggering of API calls in response to user inputs is becoming more feasible and effective. This key functionality allows LLMs to understand when to initiate a function and to formulate the necessary JSON for API calls OpenAI Documentation, and similarly featured by Anthropic.

Getting Started with Function Calling

Function calling enables LLMs to interact directly with external resources through API calls, fostering a dynamic and responsive communication model. This functionality is crucial across several applications, from conversational agents delivering real-time data updates to complex data processing tasks.

How It Works:

1. Detection: The LLM first identifies the need for external data retrieval or action based on user input.
2. Response Generation: The LLM constructs a JSON object containing the necessary parameters for the API call.
3. Execution: The API call is executed externally; the results are then processed and presented as a coherent response.

Case Studies in Function Calling with LLMs

GPT-4 Example

Imagine a scenario where a user inquires about the status of an order. The LLM pinpoints this as a cue for a function call, potentially fetching the required data from a database:

{
    "type": "function",
    "function": {
        "name": "get_product_order",
        "description": "Fetch order status",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"}
            },
            "required": ["order_id"]
        }
    }
}

This schema, when sent in an API request, helps the LLM generate the appropriate JSON to retrieve order data:

def get_completion(messages, model="gpt-3.5-turbo-1106", tools=None):
    response = openai.chat.completions.create(
        model=model,
        messages=messages,
        tools=tools
    )
    return response.choices[0].message

messages = [{"role": "user", "content": "Where's my order 231445?"}]
response = get_completion(messages, tools=tools)

The resultant output aids the LLM in efficiently communicating with an external API to procure and return the desired information.

Claude Opus Example

A similar situation where a user questions the status of an order triggers a corresponding function call via the Claude Opus model, structured as follows:

def get_completion(messages, model="claude-3-opus-20240229", tools=None):
    response = client.beta.tools.messages.create(
        model=model,
        messages=messages,
        tools=tools
    )
    return response.choices[0].message

messages = [{"role": "user", "content": "Where's my order 231445?"}]
response = get_completion(messages, tools=tools)

Key Use Cases for Function Calling in LLMs

• Conversational Agents: Power intelligent bots that fetch live data, collaborate with other services, or provide instant customer support.
• Data Extraction: Organize unstructured data or gather contextual information from multiple sources.
• API Integration: Enhance LLM functionalities by integrating diverse services and APIs.
• Natural Language Interaction: Enable models to parse queries and transform them into actionable API calls.

Conclusion

Function calling magnifies the capabilities of LLMs, transforming them from mere conversational entities to robust interfaces that connect and interact with a broader technology ecosystem. By translating natural language inputs into structured API calls, LLMs extend their role far beyond interaction, becoming integral components of digital infrastructure.

For further detailed guidance, see the official documentation:

• Google Gemini API Documentation
• Python Langchain Use Cases
• OpenAI Function Calling Guide
• Anthropic Claude Tool Use Documentation