Snowflake presents the Open Data Lakehouse

How to use Snowflake as a Data Lakehouse with Iceberg Tables

Data Warehouses, especially cloud-based ones, are becoming more and more a Data Lakehouse. Now, Snowflake has announced the shift to the Open Data Lakehouse via Iceberg Tables.

SaaS Data Warehouses vs. Data Lakehouses: Blurring the Lines

How the Cloud Data Warehouse is developing more and more into a Data Lakehouse

medium.com

Also just a few days ago, Snowflake has already promoted the support of structured types within Snowflake. With Iceberg Tables within Snowflake, you are now able to use it as the table catalog for managing metadata. In this mode, Snowflake can read and write Iceberg and Parquet data to your storage account, and it takes care of storage maintenance operations like compaction, expiring snapshots, and deleting orphan files[1][2].

Snowflake launches Structured Types

How Snowflake integrates Iceberg Tables even more into it’s Ecosystem

towardsdev.com

With this new update, Snowflake wants to integrate Apache’s Iceberg Ecosysteme even more. The Apache Iceberg’s ecosystem of diverse adopters, contributors and commercial support continues to grow, establishing itself as the industry standard table format for an Open Data Lakehouse[3].

Architecture of Snowflake and Iceberg Catalog — Image Source: Snowflake[3]

If you are using Snowflake on AWS and are already using Glue Data Catalog for your Data Lake, then the GLUE catalog integration provides an easy way to start querying those tables with Snowflake. If you are you using Snowflake on Azure or GCP and only need Snowflake as a query engine without full read-and-write, you can then integrate Snowflake using the OBJECT_STORE catalog integration[2][3].

For a deeper dive and more information on how to setup up the whole thing, please also use the provided links and sources below. This update is quite a big one and provides customers with a new approach on how to build up a (Open) Data Lakehouse with Snowflake and Iceberg. A driver for this is likely to be the competition, such as Microsoft Azure and Google, which are moving in a similar direction with solutions such as BigLake and BigQuery as well as Microsoft Fabric.

Sources and Further Readings

[1] Snowflake, SQL Updates (2023)

[2] Snowflake, Unifying Iceberg Tables on Snowflake (2023)

[3] Snowflake, Build an Open Data Lakehouse with Iceberg Tables, Now in Public Preview (2023)