How To Use Azure OpenAI With Cognitive Search in TypeScript

A workaround using Fetch

Disclaimer: This is not an introduction to Azure OpenAI & Cognitive Search

TL;DR

Are you looking to build your own version of ChatGPT? Do you want a tool that’s able to hold a conversation with your customers? Well, you’re in luck.

Microsoft has partnered with OpenAI to allow developers to use their GPT models in Azure with OpenAI Studio. Using Cognitive Search, businesses are able to connect the GPT-35-Turbo and GPT-4 models to their own data, and many are now considering using it as a chatbot for internal data.

In this blog, I’ll show you can utilize your Azure OpenAI deployment and Cognitive Search together with TypeScript, allowing you to combine the natural language skills of OpenAI with the search capabilities of Cognitive Search.

Prerequisites

Before coding it up in TypeScript, you need to make sure you’ve got a couple of things set up:

• An Azure account with an active subscription. Create an account for free.
• An Azure OpenAI (AOAI) service and model deployment. To create a deployment, simply use OpenAI Studio. Azure OpenAI requires registration and is currently only available to approved enterprise customers. You are required to submit a registration form
• An Azure Cognitive Search service (running at least on plan ‘basic’, which costs €0,10/hour.) and an index filled with some data. Find a well-written guide on inserting data here.

The Azure OpenAI client library

Microsoft released an official npm package for developers to use Azure OpenAI in Typescript. It allows easy access to your OpenAI instance and has endpoints for chat completions, completions, and embeddings.

Implementation is very simple. Create an OpenAiApi object reference your Azure resources, before you can use it to create the completion.

import { 
  Configuration, 
  OpenAIApi, 
  ChatCompletionRequestMessageRoleEnum
} from "azure-openai";

this.openAiApi = new OpenAIApi(
  new Configuration({
     apiKey: "{YOUR_AZURE_OPENAI_KEY}",
     // add azure info into configuration
     azure: {
        apiKey: "{YOUR_OPENAI_AZURE_KEY}",
        endpoint: "{YOUR_AZURE_OPENAI_ENDPOINT}",
        deploymentName: "{YOUR_OPENAI_MODEL_NAME}" // created in AOAI studio
     }
  }),
);

try {
  const completion = await this.openAiApi.createChatCompletion({
    model: process.env.AOAI_MODEL,
    messages: this.chatHistory[sender],
    temperature: 0.7,
    max_tokens: 1000,
    top_p: 0.7,
    //dataSources: NOT POSSIBLE TO QUERY COGNITIVE SEARCH
  });
  console.log("createChatCompletion response: " + completion.data.choices[0].message.content);
  } catch (error) {
    ...
  }

This example uses the ChatCompletionsendpoint of the Azure OpenAI deployment and works without any hassle. The issue occurs when we want to query our own data from Cognitive search. To do so, we need to call a different endpoint, /extensions .

Unfortunately, as of August 2023, it is not possible to query data from Cognitive Search using this library. For this reason, a different approach is necessary.

Solution: Using the Azure OpenAI ReST endpoint

Fortunately, using the official library is not necessary. Microsoft also enables developers to access their OpenAI instances using a ReST API. Find out more on this link.

To set our Cognitive Search as a data source parameter, we have to use the extensions endpoint.

POST {your-resource-name}/openai/deployments/{deployment-id}/extensions/chat/completions?api-version={api-version}

Find the specs here. When calling this endpoint, we must send the user prompt (message) with it. However, it is recommended to not only send one message but a history of messages, to make GPT aware of the context of the message.

A message in our example is of type ChatMessage and consist of the message, the prompt, and a role , which basically tells GTP who the creator of the message is. This could be the assistent , meaning GPT itself, the system or user . Find out more here.

Our first task involves creating a function to access our OpenAI deployment, by using the straightforward fetch method.

export type ChatMessage = {
  role: string;
  content: string;
};

/*
* Calls the Azure OpenAI and Cognitive Search resource and returns the completion 
* Params: msg ... Array of previous messages for the context, chat history 
*/
async fetchCompletions(msg: ChatMessage[]): Promise<Response> {
  return fetch('https://{AZURE_OPENAI_ENDPOINT/openai/deployments/{YOUR_AZURE_OPENAI_DEPLOYMENT}/extensions/chat/completions?api-version=2023-06-01-preview', {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      'api-key': '{YOUR_AZURE_OPENAI_KEY}'
    },
    body: JSON.stringify({
      messages: msg,
       dataSources: [
        {
          type: "AzureCognitiveSearch",
          max_tokens: 1000   // optional
          temperature: "0.7",   // optional
          top_p: 0.95,   // optional
          parameters: {
              endpoint: "{YOUR_COGNITIVE_SEARCH_ENDPOINT}",
              key: "{YOUR_COGNITIVE_SEARCH_KEY}",
              indexName: "{YOUR_COGNITIVE_SEARCH_INDEX}",
          },
          topNDocuments: "5"  // optional, set amount of search results
        }
      ] 
      // Optional: only available with gpt model versions >= 0613. 
      function_call: "auto",
      functions:  [ ... ]
    }),
  })
}

The keys can be found on Azure and/or in OpenAI Studio. Next, we have to use our written function and then process the call.

const messages = [ ... ];
const response: Response = await this.fetchCompletions(messages);
const data = await response.text();
const jsonData = await JSON.parse(data);  
if (response.ok) {
  const msg = jsonData.choices[0]?.messages[1]?.content; // messages[1] when using extensions endpoint, otherwise [0]
  console.log(msg)
}

Note that our endpoint returns a stream and not a simple JSON object. We can’t simply parse the response using response.json() but have to use the text() method before parsing it to a JSON.

As of now, the code is working and you’ll see results containing your internal data on Cognitive Search transformed into a readable text by OpenAI. However, the response might include references to documents and contain strings in the following format: doc[NUMBER] .

Instead of these references, we can put the link into the answer. To do so, use the following function.

if (response.ok) {
  ...
  msg = this.replaceDocNumber(msg, JSON.parse(jsonData.choices[0]?.messages[0]?.content));
  ...
}

replaceDocNumber(msg: string, citations: any) {
    const replacedString = msg.replace(/\[doc\d+\]/g, (match, index) => {
      // "match" is the entire matched string
      // find [doc{NUMBER}] in citations and replace with doc urls
      const num = parseInt(match.match(/\d+/)[0]);
      if (num > 0) {
        return citations.citations[num-1].url;
      }
    });
    return replacedString;
  }

Voilà, that’s it! Let me know in the comments if you found this tutorial helpful.