LANGCHAIN — Can AI Experiences Be Built With LLMs and the Semantic Layer?

Technology offers us a unique opportunity, though rarely welcome, to practice patience. — Allan Lokos.

Building AI experiences with LangChain, LLMs, and the semantic layer offers a powerful way to leverage the capabilities of large language models for natural language processing and data querying. In this tutorial, we will focus on Cube’s integration with LangChain and demonstrate how to build a chat-based demo application using LLMs for natural language queries to a semantic layer.

Semantic document loader

Cube’s integration with LangChain introduces the document loader, which populates a vector database with embeddings derived from the data model of the semantic layer. This vector database can then be queried to find best-matching entities of the semantic layer. This is useful for matching free-form input, such as queries in natural language, with the views and their members in the data model.

The following code snippet demonstrates how to ingest metadata from Cube and populate the vector database using the CubeSemanticLoader:

def ingest_cube_meta():
    ...
    loader = CubeSemanticLoader(api_url, api_token)
    documents = loader.load()
    ...
    with open("vectorstore.pkl", "wb") as f:
        pickle.dump(vectorstore, f)

Chat-based demo app, dissected

The chat-based demo application showcases the interaction between the user, the vector database, LLMs, and the semantic layer. Let’s break down the key components of the demo application:

1. LLM setup: Initializing the OpenAI model with the provided API key.

import ...
load_dotenv()
llm = OpenAI(
    temperature=0, 
    openai_api_key=os.environ.get("OPENAI_API_KEY"),
    verbose=True
)

1. User input and vector store initialization: Getting user input and initializing the vector store.

question = st.text_input(
    "Your question: ",
    placeholder="Ask me anything ...",
    key="input"
)
if st.button("Submit", type="primary"):
    check_input(question)
    vectorstore = init_vectorstore()

1. Querying the vector store: Querying the vector store for documents similar to the user’s question.

docs = vectorstore.similarity_search(question)
table_name = docs[0].metadata["table_name"]
column_docs = vectorstore.similarity_search(
    columns_question,
    filter=dict(table_name=table_name),
    k=15
)

1. Building the prompt and calling OpenAI: Constructing the prompt and calling the OpenAI large language model.

prompt = CUBE_SQL_API_PROMPT.format(
    input_question=question,
    table_info=table_name,
    columns_info=columns,
    top_k=1000,
    no_answer_text=_NO_ANSWER_TEXT,
)
llm_answer = llm(prompt + PROMPT_POSTFIX)

By following these code snippets and examples, you can create your own chat-based demo application that leverages the semantic layer and LLMs for natural language queries to Cube’s SQL API.

Cube’s integration with LangChain provides a seamless interface for querying data using natural language. By abstracting the complexities of SQL and leveraging the power of LLMs, it provides builders of AI experiences with a user-friendly and error-prone approach to data access.

In conclusion, this tutorial has provided a glimpse into the integration of Cube with LangChain and the possibilities it offers for building AI experiences with LLMs and the semantic layer. By leveraging the strengths of both platforms, developers can create powerful AI applications that enable natural language interactions with complex data models.