LANGCHAIN — Parallel Function Calling Extraction for Structured Data Extraction

Technology offers us a unique opportunity, though rarely welcome, to practice patience. — Allan Lokos.

Extraction is the process of extracting structured data from unstructured data. A concrete example of this is entity extraction. As Sematext explains:

Entity extraction is, in the context of search, the process of figuring out which fields a query should target, as opposed to always hitting all fields. The reason we may want to involve entity extraction in search is to improve precision.

With old function calling, you could only get one function call back at a time. This made it so that if you wanted to extract multiple pieces of information at a time you had to do some hacks. For example, let’s assume you wanted to extract the following pieces of information:

class Person(BaseModel):
    name: str
    age: int

If I just passed that directly as a function to the old OpenAI models, it would only ever return one piece of information! This is obviously not acceptable if I want to extract multiple pieces of information.

In order to extract multiple pieces of information, I would have to do a hack like:

class Information(BaseModel):
    people: List[Person]

This would construct a new object that had as a field a list of Person objects. This way, when the OpenAI model constructed the function invocation, it would construct one argument that was a list of people.

So how does parallel function calling improve this?

It removes a LOT of the complexity and hacks we had to do. You no longer have to create that ugly, extraneous Information class - you can just pass in Person and Location directly as functions, and the model will output those as separate function calls. This is an improvement because it means:

• Less logic needed to create the LLM input (better developer experience)
• Less complicated function definitions to pass to the LLM (saves on tokens, less likely to confuse the LLM)
• Less complicated output the LLM needs to produce (smaller chance the LLM messes up and outputs incorrect json)
• Less logic needed to parse the LLM output (better developer experience)

We’ve put all this together into a simple extraction chain. The full logic for the chain looks something like:

# Create a prompt telling the LLM to extract information
prompt = ChatPromptTemplate.from_messages({
        ("system", _EXTRACTION_TEMPLATE),
        ("user", "{input}")
    })

# Convert Pydantic objects to the appropriate schema
tools = [convert_pydantic_to_openai_tool(p) for p in pydantic_schemas]
# Give the model access to these tools
model = llm.bind(tools=tools)
# Create an end to end chain
chain = prompt | model | PydanticToolsParser(tools=pydantic_schemas)

Using this chain we can now do things like:

# Make sure to use a recent model that supports tools
model = ChatOpenAI(model="gpt-3.5-turbo-1106")
chain = create_extraction_chain_pydantic(Person, model)
chain.invoke({"input": "jane is 2 and bob is 3"})

And get:

[Person(name='jane', age=2), Person(name='bob', age=3)]

While most people are excited about function calling for creating agents, there is a very real (and very useful) use case around using it to structure outputs from LLMs more generically. One of these use cases is extraction, but with old function calling you had to do some hacks to get that to work. With parallel function calling it is significantly easier. Try it out!