Google has integrated Gemini into BigQuery

How you can now use BigQuery with Gemini & Vertex AI

Big news for all of us, who are using BigQuery for AI tasks. You can now create remote models based on Gemini and Vertex AI Large Language Models.

With Gemini, Google wants to catch up with Microsoft and OpenAI and now they are starting the phase to integrate it into their cloud services.

This means big news for all BigQuery users, who can now use the Data Warehouse SaaS service to create AI models. This is already generally available and you can now[1]:

• Create a remote model based on the gemini-pro Vertex AI large language model (LLM).
• Use the ML.GENERATE_TEXT function with a remote model based upon gemini-pro to perform generative natural language tasks on text stored in BigQuery tables.
• Use the BigQuery DataFrames GeminiTextGenerator class in the bigframes.ml.llm module to create estimator-like Gemini text generator models.

The Gemini Pro version is designed to handle natural language tasks, multiturn text and code chat, and code generation. For all, who want to use Vertex AI not only in the context of BigQuery, Google also offers Vertex AI with the following foundation model APIs[2]:

• Gemini API (Multimodal data, text, code, and chat)
• PaLM API (Text, chat, and embeddings)
• Codey APIs (Code generation, code chat, and code completion)
• Imagen API (Image generation, image editing, image captioning, visual question answering, and multimodal embedding)

For a deeper dive and learning how to create models, please also use my other articles linked down below. Another great AI feature that Google has launched lately for BigQuery is the possibility to do Vector indexing and searching. That makes BigQuery now also capable for doing vector database tasks.

Then remembering that Google BigQuery a few years ago was “just” only a column based Data Warehouse and offered some ML functions through BigQueryML, it now quickly became more and more a Data Lakehouse which unifies the traditional Data Warehouse with NoSQl and Vector database features, but now also can be used with powerful AI tools like Gemini & Co. This is clearly an advantage because as a company and team, you won’t need that many other services and can use BigQuery as your hub for analytics.

Sources and Further Readings

[1] Google, BigQuery release notes (2024)

[2] Google, Model information (2024)