Google improves Data Security in it’s Data Warehouse BigQuery

Using column level SQL encryption with Cloud KMS keys

Whether to meet national or international guidelines (e.g. the DSGVO in Europe) or to protect data within the company, it helps to encrypt it accordingly. And this is exactly where Google comes in with its’ new features within the Data Warehouse BigQuery.

Here is the full statement of Google about this new feature: Deterministic encryption SQL functions are now generally available (GA). New AEAD encryption functions include DETERMINISTIC_ENCRYPT, DETERMINISTIC_DECRYPT_BYTES, and DETERMINISTIC_DECRYPT_STRING. These functions allow column-level encryption and decryption of data while supporting aggregation and table joins. — Google [1]

So from now on, you can use beside the Cloud Key Management Service to encrypt the keys, which in turn encrypts the values in BigQuery tables also creates a second layer of protection at the column level using AEAD encryption capabilities. This additional layer of protection encrypts the Data Encryption Key (DEK) with a second Key Encryption Key (KEK). By referencing an encrypted key set in BigQuery instead of a plaintext key set, the risk of key visibility is reduced. The KEK is a symmetric encryption key set that is securely stored in the Cloud Key Management Service (KMS) and managed with Identity and Access Management (IAM) roles and permissions.[2]

In line with this, Google had already recently improved the interaction between BigQuery and the Data Catalog, in which the management of encryption can be controlled. Read in this article more about it.

After Google has recently equipped BigQuery with many new features, one of the biggest was probably the new feature BigLake, which allows companies to realize even BigQuery SQL analysis across platforms. Google now follows in the area of security. This is the only way that companies can use the functionality at all. The number of hacker attacks is increasing, and at the same time, authorities are also requiring companies to secure their data, as otherwise they could face heavy penalties. Companies that do not secure their data appropriately, whether against external attacks or internally only make data readable for the people for whom it is intended, otherwise run a great risk. In addition to the features Data Catalog and Column based encryption, Google has also improved in the area of connection security.

When using BigQuery Omni you can now use the Cloud console to set up VPC service control perimeters to restrict access from BigQuery Omni to external clouds [1].

So it seems that Google really takes privacy seriously and offers customer solutions here. Companies should be happy about this, because I know myself that use cases often fail because data protection cannot be observed. So there are new possibilities here, and these features should also simplify the whole issue of data governance.

Sources and Further Readings

[1] Google, BigQuery Release Notes (2022)

[2] Google, SQL column-level encryption with Cloud KMS keys (2022)