Meet LMQL: An Open Source Query Language for LLMs

Developed by ETH Zurich, the language explores new paradigms for LLM programming.

I recently started an AI-focused educational newsletter, that already has over 160,000 subscribers. TheSequence is a no-BS (meaning no hype, no news, etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers, and concepts. Please give it a try by subscribing below:

In the realm of technology, large language models(LLMs) have exhibited exceptional capabilities across diverse tasks, including question answering and code generation. At its core, a LLM excels in automatically generating coherent sequences based on given inputs, relying on statistical likelihood. Leveraging this ability, users can prompt these models with language instructions or examples, enabling the execution of various downstream tasks. The advanced techniques of prompting can even facilitate interactions involving the language model, users, and external tools like calculators. However, achieving state-of-the-art performance or tailoring language models to specific tasks often necessitates implementing complex, task-specific programs, which may still rely on ad-hoc interactions.

Language Model Programming (LMP) is an emerging discipline that is gaining traction to tackle those challenges. LMP represents a significant advancement in language model prompting, transitioning from pure text prompts to an intuitive combination of text prompting and scripting. Furthermore, LMP empowers users to specify constraints on the language model’s output, facilitating effortless adaptation to a multitude of tasks while abstracting the inner workings of the language model and providing high-level semantics. Recently, researchers from the prestigious ETH Zurich released LMQL, a query language for LLMs that builds on the principles of LMP.

Conceptually, combines declarative SQL-like elements with an imperative scripting syntax. By leveraging an underlying runtime that seamlessly integrates with existing language models (LMs), LMQL offers an accessible solution that requires a simple adjustment in the decoder logic. This compatibility ensures easy adoption and support for LMQL, eliminating the need for complex changes or migrations.

Inside LMQL

LMQL, a declarative programming language inspired by SQL, has been designed specifically for seamless interaction with language models. An LMQL program comprises five essential parts, each playing a crucial role in shaping the query’s behavior. These components include the decoder, the query itself, the from clause specifying the targeted model, the where clause defining constraints, and finally, a distribution instruction. Let’s delve into each of these elements:

1) The Decoder

The decoder, specified as a string, determines the decoding algorithm employed during text generation. It dictates how the model’s output is transformed into meaningful results. The choice of decoder significantly influences the generated text’s quality, diversity, and overall performance.

2) The Query

The query block serves as the core interaction mechanism with the language model. Conceptually, it resembles the body of a Python function, albeit with certain restrictions and additions. Inner function declarations are disallowed, although imports can be included. Each top-level string within the query block is treated as a direct query to the language model. These query strings support two special escaped subfields, akin to Python f-strings.

“{varname}” enables the retrieval of a variable’s value from the current scope.

“[varname]” represents a phrase, also known as a hole, that will be generated by the language model. When the model produces values for these holes, they are subject to the constraints defined in the query’s where clause. The decoding procedure, specified by the decoder, is applied within the context of these constraints. Upon completion, a corresponding variable is created within the query program’s scope and assigned the generated value. If a variable with the same name already exists, it will be overwritten.

3) The Model and Where Clauses

In an LMQL program, the from clause specifies the model being queried. This crucial component defines the underlying language model used for text generation. The where clause, on the other hand, allows developers to define constraints that influence the generated output. These constraints impose specific limitations or conditions that the language model must adhere to, ensuring the desired qualities in the generated text.

4) Distribution Instructions

The distribution instruction plays a vital role in the LMQL program by guiding the distribution of generated values. It defines how the generated results are distributed and presented, offering control over the outcome’s format and structure.

By understanding the intricacies of these components, tech-savvy individuals can effectively harness the power of LMQL to interact with language models, enabling sophisticated text generation while incorporating constraints and preserving control over the decoding process.

Using LMQL

LMQL extends the capabilities of natural language prompting by integrating Python into its framework. This integration empowers developers to express natural language prompts that contain both text and code, enhancing the flexibility and expressiveness of the queries. Additionally, LMQL allows users to define high-level constraints on the language model’s output. These constraints are automatically translated into token-level prediction masks, which actively influence the text generation process. This ensures that the resulting text aligns with the specified constraints.

To facilitate query development, LMQL provides a dedicated Playground IDE. Within this environment, users can inspect the interpreter’s state, validation results, and model outputs at any point during text generation. This includes the ability to analyze and explore different hypotheses generated during beam search, offering valuable insights for fine-tuning the language model’s behavior.

Integrating LMQL queries into your Python codebase is remarkably straightforward. By employing the lmql.query decorator and supplying the query code as a multi-line string, you can effortlessly define and execute LMQL queries within Python.

To begin, simply decorate a Python function of your choice with the lmql.query decorator. This straightforward step enables the function to be automatically compiled into an LMQL query. The query code, specified within a multi-line string, defines the desired behavior and interactions with language models.

Once decorated, the function retains its original signature and can be invoked with the same arguments as before. The seamless integration allows for a smooth transition from traditional Python functions to LMQL-powered queries.

Executing the decorated function triggers the LMQL query’s execution. The return value of the decorated function holds the result of the LMQL query. This means that you can conveniently access and utilize the outcome of the query within your Python codebase.

By leveraging the power of decorators, you can effortlessly incorporate LMQL queries into your Python projects. This streamlined integration empowers developers to take advantage of LMQL’s querying capabilities while maintaining the familiarity and versatility of Python functions.

import lmql
import asyncio

@lmql.query
async def hello():
    '''lmql
    argmax
        "Hello[WHO]"
        return WHO
    from
        "openai/text-ada-001"
    where
        len(TOKENS(WHO)) < 10
    '''

# run 10 instances of 'hello' in parallel
await asyncio.gather(*[hello() for _ in range(10)])

In summary, LMQL introduces a tech-friendly approach to interact with language models efficiently. By leveraging its declarative nature, developers can effectively control and guide the generation of text, making it a valuable tool for various applications in the tech industry.