avatarAshhadul Islam

Summary

The article demonstrates how to use the NSQL-Llama-2–7B model to generate SQL queries from free-flowing text instructions, specifically to find the names of singers who performed at a given concert.

Abstract

The article provides a detailed guide on how to use the NSQL-Llama-2–7B model to generate SQL queries from free-flowing text instructions. It begins by stating the goal of finding the names of singers who performed at a specific concert using a given table schema. The software requirements for this task include Ubuntu, Python version 3.10 or above, and the NSQL-Llama-2–7B model from Hugging Face. The hardware requirements include at least 32 GB RAM. The article then provides two versions of code that demonstrate how to pass the schema of the table and the query in plaintext to the bot, which then generates the SQL query. The first version of the code shows the schema of four tables and asks the bot for the name of the singer who performed at concert_id 12. The second version of the code extracts table details from the database as a string and passes it as part of the prompt. The article notes that both pieces of code take a long time to run, and suggests that this method could be useful if the compute time can be reduced.

Bullet points

  • The goal is to find the names of singers who performed at a specific concert using a given table schema.
  • The software requirements include Ubuntu, Python version 3.10 or above, and the NSQL-Llama-2–7B model from Hugging Face.
  • The hardware requirements include at least 32 GB RAM.
  • The article provides two versions of code that demonstrate how to pass the schema of the table and the query in plaintext to the bot, which then generates the SQL query.
  • The first version of the code shows the schema of four tables and asks the bot for the name of the singer who performed at concert_id 12.
  • The second version of the code extracts table details from the database as a string and passes it as part of the prompt.
  • Both pieces of code take a long time to run, and the article suggests that this method could be useful if the compute time can be reduced.

Super Quick: LLAMA2 on CPU Machine to Generate SQL Queries from Schema

Using NSQL-Llama-2–7B, to generate SQL queries from free-flowing text instruction.

Goal:

Given a table schema as follows:

CREATE TABLE stadium (
    stadium_id number,
    location text,
...
)

CREATE TABLE singer (
    singer_id number,
    name text,
    country text,
   ...
)

CREATE TABLE concert (
    concert_id number,
    concert_name text,
    ...
)

CREATE TABLE singer_in_concert (
    concert_id number,
    singer_id text
)

We want to know the names of the singers who performed at, say, concert_id 12.

User: Using valid SQLite, answer the following questions for the tables provided above. What is the name of the singer who performed at concert_id 12 ? Bot: SELECT T2.name FROM singer_in_concert AS T1 JOIN singer AS T2 ON T1.singer_id = T2.singer_id WHERE T1.concert_id = 12

The bot is able to formulate complex join queries. Let us see how we can do this.

Software Requirements:

Operating System: Ubuntu preferably

Python: version 3.10 and above

Model:https://huggingface.co/NumbersStation/nsql-llama-2-7B (To be downloaded by code). This model is based on Meta’s original Llama-2 7B model and further pre-trained on a dataset of general SQL queries and then fine-tuned on a dataset composed of text-to-SQL pairs.

Hardware Requirements:

Running on CPU: At least 32 GB RAM.

Almost 26 GB out of 32 GB used up while loading the model

As the model is not quantized, it takes up ~26 GB RAM during execution.

Packages required:

torch
transformers

Version 1:

In this version we pass the schema of the table and our query (in plaintext) to the bot.

The bot in return, gives us the SQL query.

The provided code below demonstrates how we can provide create commands for tables and then request the bot to generate a query that retrieves the desired information.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("NumbersStation/nsql-llama-2-7B")
model = AutoModelForCausalLM.from_pretrained("NumbersStation/nsql-llama-2-7B", torch_dtype=torch.bfloat16)

text = """CREATE TABLE stadium (
    stadium_id number,
    location text,
    name text,
    capacity number,
    highest number,
    lowest number,
    average number
)

CREATE TABLE singer (
    singer_id number,
    name text,
    country text,
    song_name text,
    song_release_year text,
    age number,
    is_male others
)

CREATE TABLE concert (
    concert_id number,
    concert_name text,
    theme text,
    stadium_id text,
    year text
)

CREATE TABLE singer_in_concert (
    concert_id number,
    singer_id text
)

-- Using valid SQLite, answer the following questions for the tables provided above.


-- Give What is the name of the singer who performed at concert_id 12 ?

SELECT"""

input_ids = tokenizer(text, return_tensors="pt").input_ids

generated_ids = model.generate(input_ids, max_length=500)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

Above piece of code shows the schema of 4 tables — stadium, singer, concert and singer_in_concert. It then asks the bot, what is the name of the singer who performed at convert_id 12. The bot should be able to create a join query and give the response.

Output of the chatbot

Version 2

Often it is difficult to pass the table schema as string. It would be good if we could have a piece of function that extracts table details from our database as a string and passes that as part of the prompt.

The code for that is as follows.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
import subprocess

def get_sql_create_queries(host,database,user,password.port):
 # Command to execute pg_dump
 command = [
     "pg_dump",
     f"--host={host}",
     f"--dbname={database}",
     f"--username={user}",
     f"--port={port}",
     "--no-owner",
     "--no-privileges",
     # "--data-only",
     "--schema-only",
     "--no-comments",
 ]

 try:
     output = subprocess.check_output(command, stderr=subprocess.STDOUT, text=True)
     # Extract the create table commands Only
     test_sub="CREATE TABLE"
     res = [i for i in range(len(output)) if output.startswith(test_sub, i)]
     print(res)
     create_queries=""
     for start_point in res:
      subs=output[start_point:]
      test_sub=";"
      semi_colons=[i for i in range(len(subs)) if subs.startswith(test_sub, i)]
      just_create_query=subs[:semi_colons[0]]
      create_queries+=just_create_query+"\n"
     print(create_queries)
     return create_queries

 except subprocess.CalledProcessError as e:
     print("Error:", e.output) 
     return ""



tokenizer = AutoTokenizer.from_pretrained("NumbersStation/nsql-llama-2-7B")
model = AutoModelForCausalLM.from_pretrained("NumbersStation/nsql-llama-2-7B", torch_dtype=torch.bfloat16)

# Database connection parameters
host = "localhost"
database = "ashhadulislam"
user = "ashhadulislam"
password = "ashhadulislam"
port = "5432"

create_queries=get_sql_create_queries(host,database,user,password.port)
text=create_queries+'''

-- Using valid SQLite, answer the following questions for the tables provided above.


-- Give What is the name of the singer who performed at concert_id 12 ?

SELECT
'''

input_ids = tokenizer(text, return_tensors="pt").input_ids

# generated_ids = model.generate(input_ids, max_length=500)
generated_ids = model.generate(input_ids, max_length=len(text))

# print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
output = 'SELECT' + output.split('SELECT')[-1]
print(output)

In this scenario, the code initially retrieves all the create table commands related to the selected database and incorporates them into the prompt, which subsequently asks for an SQL command.

Output:

SELECT T2.name FROM public.singer_in_concert AS T1 JOIN public.singer AS T2 ON T1.singer_id = T2.singer_id WHERE T1.concert_id = 12
Output came after 20 minutes

Note: Both the pieces of code take a LONG time to run. For me it was 20 minutes.

I believe this can be useful if the compute time can be reduced. Your comments are welcome.

Following are my other llm related articles:

Open Source LLM:

PDF Related

Super Quick: Retrieval Augmented Generation (RAG) with Llama 2.0 on Company Information using CPU

Super Quick: Fine-tuning LLAMA 2.0 on CPU with personal data

Database Related

Super Quick: LLAMA2 on CPU Machine to Generate SQL Queries from Schema

Close Source LLM (OpenAI):

PDF Related

Super Quick PDF-based ChatGPT Tutorial in Python

Chatbot Document Retrieval: Asking Non-Trivial Questions

Database Related

Super Quick: Connecting ChatGPT to a PostgreSQL database

In Plain English

Thank you for being a part of our community! Before you go:

Llama 2
Large Language Models
Sql
Database
ChatGPT
Recommended from ReadMedium