The Ultimate Guide to GCP’s Logging Query Language

How to build queries that analyze Cloud Logging data.

A Meta Query Language

LQL can be used in the Logs Explorer to fetch real-time data on Google Cloud products like Cloud Functions and Virtual Machines as well as non-GCP resources like resources connected to Amazon Web Services’ accounts.

If you’ve never accessed the Logs Explorer within your GCP project, you can navigate to the Logs Explorer in the GCP console and then build queries either manually or by clicking to enable various filters.

If you are simply seeking broad information on resources like cloud functions, then it’s likely that your activity will be confined to the Logs Explorer UI and the query pane.

The query pane enables click-based operations such as:

• Searching for text strings across specific fields
• Toggling options from within the filter menus
• Composing more advanced queries using LQL
• Query execution operations (view, run, edit, save, etc.)

Building Logging Queries

Once your queries become more specific, it is likely you’ll need to move beyond the constraints of the LQL point-and-click UI features.

Thankfully, Google provides a guide to LQL operations.

Reading this guide, you’ll notice that the operations include SQL-like syntax and functionality.

Google, itself, notes that LQL operations can be thought of in two ways: Queries and comparisons.

By the GCP definition, a query is any line of code that includes an expression. A comparison is either a single value or a more conventional Boolean comparison.

Pardon the interruption: For more Python, SQL and cloud computing walkthroughs, follow Pipeline: Your Data Engineering Resource.

To receive my latest writing, you can follow me as well.

Search a String

Here is a string-based comparison that searches for compute instances in a specific project.

resource.type = "gce_instance" AND resource.labels.project_id = "project_one"

Note the similarities to a SQL query, which might read as the following.

SELECT resource FROM metadata
WHERE resource.type = "gce_instance" AND resource.labels.project_id = "project_one"

Operators

Like SQL, LQL also supports comparison operators that enable querying of Boolean, numeric and string-based values.

=           # equal
!=          # not equal
> < >= <=   # numeric ordering
:           # "has" matches any substring in the log entry field
=~          # regular expression search for a pattern
!~          # regular expression search not for a pattern

If I wanted to search the logs for any instance that is triggering an error and I wanted to ensure error was in the message, I would use the ‘=~’ operator.

resource.type = 'gce_instance' 
jsonPayload.message =~ "Error"

Conditional Filtering

Like SQL, I can use regEx searches to filter multiple conditions.

--Searching for instances whose payload messages include both error and failure.

resource.type = 'gce_instance'
jsonPayload.message =~ "Error" AND jsonPayload.message =~ "failure"
severity = ERROR

--Searching for instances whose payload messages include either error or restarting.

resource.type = 'gce_instance' 
jsonPayload.message =~ "Error" OR jsonPayload.message =~ "restarting"
severity = ERROR

Complex Searches

We can also chain conditions within parentheses to get even more granular with our searches.

resource.type = 'gce_instance'
(jsonPayload.message =~ 'Error' OR jsonPayload.message =~ 'restarting' AND jsonPayload.message != 'exited with code')

Translating to SQL, this would be the equivalent of the following.

SELECT resource FROM metadata
WHERE message LIKE '%Error% 
OR message LIKE '%restarting%'
AND message NOT LIKE '%exited with code%'

RegEx Searches

You can also employ regex syntax to include or exclude certain entries.

We covered standard searches using the regex search string for and exclude.

Below are some examples that involve more specific regex filtering using LQL.

searchlabels.subnetwork_name =~ "(?i)Subnet" # Case insensitive

marksjsonPayload.message =~ "label=\"project.*\"" # Quotation search

labels.pod_name =~ "(error|failure)" # BOOL expression

logName =~ "/my%2Proj$" # Query using anchors

Severity

When filtering logs, you can assign flags to correspond with degrees of error severity.

For resources that enable GCP logging, these are the various levels you can filter on.

DEFAULT 
INFO
WARNING
ALERT 
ERROR
CRITICAL
DEBUG

Note: If you do not have GCP logging enabled within a resource, the base logging level will be specified as DEFAULT.

Otherwise, if you are using a Python client, you can specify the default logging level globally when instantiating and configuring your logging client.

Time Filtering

If you are creating a logging query to monitor organizational resources, it is likely that you will want to retrieve data within a certain time frame.

Using both the UI and the Logging client, it is possible to specify time ranges for LQL entries.

It is important to note that the timestamps in LQL follow the ISO format.

YYYY-MM-DDT00:00Z

This is especially relevant when attempting to filter in Python, as it is more typical to convert time variables from a timestamps or datetime objects.

Applying this format to an LQL query yields the following examples.

-- Timestamp following ISO format using >= operator.

timestamp >= "2022-05-10T09:00:00Z"

-- Timestamp following ISO format using <= operator (note military time). 

timestamp <= "2022-05-10T17:00:00Z"

-- Timestamp searching a specific date range; time nonspecific. 

timestamp >= "2022-05-10" AND timestamp <= "2022-05-13"

Establishing a time filter within an LQL query will not only ensure the accuracy and relevancy of results, it may also help optimize the query since it is demanding a defined number of results.

Take Away: Know Your Metadata

Understanding where exactly your errors are being reported is crucial when attempting to detect and preempt errors that are actually relevant to your analysis.

For instance, an issue could be flagged as severity level ERROR, but not be considered a catastrophic event that would result in a malfunctioning resource.

The severity example emphasizes what I’ve found to be a crucial component of working with logs data.

More so than any other query language, it is imperative that you know your data and you know exactly what you’re looking for.

Like SQL, if you are not explicit about exactly the resources and other parameters you are searching, the results will be overwhelming.

If you are using the Python client, in my experience, simply specifying a DEFAULT severity level or a severity level without other parameters will more than likely result in an excessive read requests error.

LQL is much simpler than SQL as a query language, however, its application is only as powerful as those who take the time to understand their metadata.

Create a job-worthy data portfolio. Learn how with my free project guide.