Using BigQuery and Google Analytics Hub

Summary

Google's Analytics Hub enhances BigQuery by enabling secure data sharing and analytics asset exchanges across organizations, supporting a data mesh approach and fostering data-driven companies.

Abstract

Google has introduced Analytics Hub, a service integrated with BigQuery, to facilitate the creation of secure data exchanges. This platform allows data providers to publish datasets that can be accessed by subscribers, addressing data reliability and cost challenges. Analytics Hub leverages BigQuery's separation of compute and storage, enabling publishers to share data without duplication and subscribers to pay only for queries they run. The service supports sharing various BigQuery objects, including tables, views, and ML models, and aligns with the principles of a Data Mesh architecture, promoting scalable and efficient data distribution within and across organizations.

Opinions

The author views Analytics Hub as a "really useful new feature" that is easy to use and scalable.
Analytics Hub is seen as great support for the Data Mesh approach, aiding companies in becoming more data-driven.
The service is considered to provide significant support for distributing data and models, such as those from BigQuery ML, within a company.

How to work with Google Analytics Hub

Great news! Google now offers the Analytics Hub for its’ cloud Data Warehouse BigQuery:

Analytics Hub is a new service in BigQuery that lets you create secure data exchanges and share analytics assets within and across organizations. This platform allows data providers to publish listings that reference shared datasets. Analytics Hub subscribers can then view and subscribe to these listings. — Google [1]

How does it Work?

Analytics Hub is a data exchange service that enables you to efficiently and securely share data assets across organizations to address data reliability and cost challenges.

Configure an Exchange in Analytics Hub — Image by Author

Before you can set up an exchange via a dataset as above, you still need to enable the API. After that, you can set up the region and also permissions and a description to share it with your company or external partners.

The following BigQuery objects can be shared using Analytics Hub [2]:

Tables

Views

Materialized views

Authorized views

Table snapshots

External tables

BigQuery ML models.

Architecture

The Google Analytics Hub is built on BigQuery and the publish and subscribe model of BigQuery datasets. The separation of compute and storage in BigQuery’s architecture enables data publishers to share data with as many subscribers as they want without having to make multiple copies of the data. Publishers are only charged for data storage, whereas subscribers only pay for queries that run against the shared data. The publisher and subscriber workflows in Analytics Hub are explained in detail in the following sections [2].

Architecture of the Analytics Hub — Source: Google [3]

Summary

I think it’s a really useful new feature or API built on top of BigQuery that is super easy-to-use and scalable. With this function, you can easily distribute data and models from e.g. BigQuery ML (Read here more about BigQuery ML) in your company. This is a great support for the Data Mesh approach and to become a data-driven company.

Using BigQuery and Google Analytics Hub

How to work with Google Analytics Hub

How does it Work?

Architecture

Summary

Sources and Further Readings