Why BigQuery Omni is a Big Deal

Google Cloud’s bet on an open platform is starting to materialize with Anthos and BigQuery Omni.

Three years ago, I started (and sadly never finished) a series called Platform Wars (Part I, Part II), evaluating the tech giants and their strategies in the age of artificial intelligence. In my piece on Google, I explained Google’s shift to an AI-first company, and why Kubernetes was a crucial part of Google’s strategy to compete in the enterprise cloud market. Fast forward two years, Google Cloud reported meaningful growth, but still stood a distant third to AWS and Azure. Thomas Kurian, a former Oracle exec, was brought in to replace Diane Greene, carrying with him a vision for a multi-cloud strategy. Then, a year ago, Google Cloud introduced Anthos, a Kubernetes-based, open platform to extend Google’s cloud services to hybrid (i.e. on-prem, multi-cloud) environments. It materialized Kurian’s vision into a product, which Ben Thompson noted in his post: “Google Cloud Next, Athos, Google Cloud and Open Source.”

Just last week, Google announced BigQuery Omni, a multi-cloud analytics solution to run BigQuery across Google Cloud, AWS, and Azure (coming soon). Now customers can use the same BigQuery UI or API to run SQL queries and build BigQuery ML models regardless of where the data is stored. More importantly, BigQuery Omni runs on Anthos and reveals Kurian’s — and Google Cloud’s — strategy to grow its addressable market. Initially, Anthos was “simply” a hybrid and multi-cloud application platform, leveraging its strong Kubernetes backbone to migrate on-prem and existing AWS/Azure applications onto GCP. With BigQuery Omni, Google is attempting to commoditize cloud infrastructure as a whole and use Anthos as a middleware to win market share. In essence, Google is betting on its superior container and AI/ML technology to compete in a growing multi-cloud world as it did with search to commoditize the underlying OS in the Internet era.

BigQuery Omni Deep-dive

Before jumping to the strategic implications of BigQuery Omni, let’s take a deeper look at how BigQuery Omni works. Unlike AWS Redshift, BigQuery decouples storage and compute (similar to how Snowflake works), taking advantage of cheaper storage costs and charging users separately for the processed data. This architectural decision makes BigQuery Omni a natural extension of that concept.

Previously, BigQuery was limited to data stored in Google Cloud. Although Google acquired cloud migration startups like Velostrata and Alooma over the years to facilitate moving data from other cloud platforms, for most enterprises the switching costs were still too high to justify using BigQuery over AWS Redshift or Azure Data Warehouse regardless of the developer experience, ease of use, or additional features.

Now BigQuery Omni runs on Anthos inside AWS to directly access data in S3 and other databases. Since BigQuery separated compute and processing with storage to start, it can now treat S3 as if it’s like data stored in GCS and run analytics on multiple clouds. The big advantage here is reduced cost in network egress charges and the removal of data migration burden to use BigQuery in the first place. With Anthos, BigQuery Omni acts almost as an AWS Marketplace solution running analytics inside AWS natively.

Strategic Implications

AWS continues to lead the $100 billion cloud market by a wide margin with Microsoft Azure carving out its space with the acquisition of Github to appeal to developers and winning the $10 billion Pentagon JEDI cloud contract. This has squeezed Google Cloud into a distant third place, even given a growing cloud market. The bright side for Google — and perhaps the justification to the Anthos and BigQuery Omni strategy — is that more companies are embracing multi- and hybrid cloud solutions.

A recent Gartner research survey on cloud adoption revealed that more than 80% of respondents using the public cloud were using more than one cloud service provider.

- Gartner, The Future of Cloud Data Management Is Multicloud

Market leaders like AWS and Azure have no motivation to pursue a multi-cloud product. Their goal is to gobble up as much of the market and lock them into their cloud platform. Google, on the other hand, has no choice and is better positioned to play the middleware card and move up the stack. Although Google owns massive infrastructure of its own to power search, maps, and email for billions of users, rather than attempting to steal enterprise customers away from the incumbent giants, it has decided that offering multi-cloud services and treating existing data siloes as potential data sources for its products would be more profitable and competitive.

From this perspective, Google Cloud’s competition may really be IBM rather than AWS and Azure. IBM acquired Red Hat for $34 billion in 2019, betting on the same open, hybrid cloud strategy with OpenShift, a popular enterprise Kubernetes platform. Google clearly has an advantage in Kubernetes as the creators (not to mention its 15+ years experience of running Borg, Google’s original container management system that Kubernetes is based on) and continues to widen its lead with active contributions to Kubernetes, Istio, and container technology. Combined with Kurian’s experience running the Fusion Middleware product at Oracle, Google seems well-positioned to grow Anthos and BigQuery Omni as the next massive scale middleware product in the cloud.

So what’s next for Anthos? The obvious answer is extending the product line to support other databases: Cloud SQL, Dataproc, BigTable, and Spanner. Personally, I’m more interested in how Google uses Looker to entice users looking for an alternative solution to AWS Quicksight or Azure PowerBI. The other interesting avenue is extending Firebase for mobile development and leveraging the existing ecosystem to expand the “middleware” market. Finally, the big question is whether or not this strategy will also accelerate the widespread adoption of AI/ML technologies. Google is widely regarded as a leader in this space, and integrating BigQuery Omni with its existing AI Platform products (i.e. kubeflow, TensorFlow, AI hub/managed Jupyter notebooks, and Kaggle) may be the final piece needed to help enterprise companies adopt AI/ML.

This isn’t to say that Google Cloud is without competition. AWS Outposts and Azure Stacks provide similar functionality to run their respective infrastructure on hybrid environments. I also wrote about how that same narrative is playing out in the IoT space with AWS Wavelength, Azure Edge Zone, and Anthos for Telecom. Finally, we also have SaaS companies like Snowflake and MongoDB focusing on multi-cloud database technology with true no vendor lock-in. Only time will tell if Google’s strategy to move up the stack will be successful or be marked as yet another futile attempt to dethrone AWS.