LANGCHAIN — Async API

The function of good software is to make the complex appear to be simple. — Grady Booch

Async API, or asynchronous API, is a programming interface that allows for non-blocking I/O operations to be performed. This means that while a program is waiting for a particular task to complete, it can continue to make progress on other tasks.

In LangChain, async support has been introduced using the asyncio library. This library uses coroutines and an event loop to enable non-blocking I/O operations. Here’s a brief overview of how async support can be used in LangChain.

Motivation

LangChain applications often involve I/O and network-bound tasks, such as calling LLM APIs and interacting with data stores. By using asyncio, you can run LLMs, chains, and agents concurrently, allowing for more efficient use of resources and faster execution of tasks.

Usage

Async support has been implemented for various components in LangChain:

LLM

# Async LLM Example
import langchain

async def main():
    result = await langchain.agenerate("OpenAI", input_data)

# Run the async function
import asyncio
asyncio.run(main())

Chain

# Async Chain Example
import langchain

async def main():
    result = await langchain.arun("LLMChain", input_data)

# Run the async function
import asyncio
asyncio.run(main())

Agent and Tool

# Async Agent and Tool Example
import langchain

async def main():
    result = await langchain.arun("SerpAPIWrapper", input_data)

# Run the async function
import asyncio
asyncio.run(main())

Up Next

Future updates to LangChain’s async support may include:

• Async support for more LLMs, Chains, and Agent tools
• Ability to run multiple tools concurrently for a single action input
• Async support for callback handlers
• More seamless support with tracing

By leveraging async API in LangChain, you can create more efficient and responsive applications, allowing for concurrent execution and improved performance.