avatarChristianlauer

Summary

Google has updated BigQuery to allow users within the same project to utilize cached results from identical queries, potentially reducing costs significantly, especially for those using BI tools like Google Looker Studio.

Abstract

Google's recent update to BigQuery introduces a cost-saving feature that enables sharing of cached query results among users in the same project. This means that if multiple users run the same query, BigQuery will serve the cached results from the initial query, thus saving the cost of running subsequent identical queries. The feature is particularly beneficial for teams using Business Intelligence (BI) tools, as these often generate the same queries for reports and dashboards. The cached results are stored in a personal anonymous dataset for 24 hours. This update is a boon for companies with many users accessing the same data, as it can lead to substantial savings, especially for large result sets. However, this feature is exclusive to the Enterprise and Enterprise Plus editions of BigQuery, which may require organizations to evaluate the cost-effectiveness of upgrading to these editions versus continuing with their current plans.

Opinions

  • The update is seen as a significant improvement for users of Google's BigQuery, particularly for those in roles such as Data Engineers, Data Chiefs Officers, and CIOs.
  • The feature is especially advantageous when using BI tools like Google Looker Studio, where the same queries are frequently used for reports and dashboards.
  • The cost savings are emphasized, suggesting that the feature could lead to considerable reductions in expenses, particularly for queries that produce large result sets.
  • There is a mention of a "Big BUT," indicating that while the feature is useful, its availability only in the Enterprise and Enterprise Plus editions could be a limiting factor for some users.
  • The article implies that organizations should perform a cost-benefit analysis to determine if upgrading to the Enterprise editions to access this feature is financially sensible.
  • The article suggests that there are additional tips and tricks for saving money in BigQuery, directing readers to further resources on the topic.

Google launches an Update for BigQuery that can reduce Costs big Time

How you can now use cached Results from the same Query issued by other Users in the same Project

Photo by Sasun Bughdaryan on Unsplash

Using cached results for users in BigQuery has been available for ever now, Google has now updated this in a way so that you can save a lot of money.

You can now use cached results from the same query issued by other users in the same project[1].

So if you and your colleague query the exact same data within BigQuery, you would save one query. Of course, now many would argue that happens not often, but when working with BI tools like Google Looker Studio it will look different. Here, often users use the same report or dashboard and will generate the same queries in the background.

The cached result is copied into the personal anonymous dataset and is kept there for 24 hours from when the query has been run. The same limits and exceptions for single-user caching apply to cross-user caching[2].

So this could save you as a company a lot of money when having many users using the same dashboard, so that BigQuery only queries the data only once. You will save even more data if the query produces a big result. So not only good news for Data Engineers and Data Chiefs Officers but also for the CIO.

But now the Big BUT: Only when you use the Enterprise or Enterprise Plus edition, you are able to make use of this feature….. So you maybe need to calculate what would be cheaper for you — using this feature or buy the Enterprise edition.

Some other tips and tricks you can use to save money can be found in the article below:

Sources and Further Readings

[1] Google, BigQuery release notes (2023)

[2] Google, Using cached query results (2023)

Data Science
Technology
Google
Bigquery
Finance
Recommended from ReadMedium
avatarShubham Verma
Iceberg Ahead! BigQuery Charts a New Course with Open Table Format — Part 2

In this second article on BigQuery Iceberg tables, we delve into advanced features that make BigQuery a powerful tool for data engineering…

4 min read
avatarAbdur Rahman
Python is No More The King of Data Science

5 Reasons Why Python is Losing Its Crown

6 min read
avatarVolker Janz
A Definitive Guide to Using BigQuery Efficiently

Make the most out of your BigQuery usage, burn data rather than money to create real value with some practical techniques.

20 min read
avatarMatt Bentley
My Favourite Software Architecture Patterns

Exploring my most loved Software Architecture patterns and their practical applications.

10 min read
avatarJessica Stillman
Jeff Bezos Says the 1-Hour Rule Makes Him Smarter. New Neuroscience Says He’s Right

Jeff Bezos’s morning routine has long included the one-hour rule. New neuroscience says yours probably should too.

4 min read
avatarAndre Abi Awad
The Build vs. Buy Equation

If you’re wondering what your business might be worth to an acquirer, there is a simple calculation you can use.

3 min read