How to Connect Google Drive to Python using PyDrive

The Ultimate PyDrive Tutorial

Adapted from a photo by Florian Olivo on Unsplash

PyDrive is a high-level Python wrapper for the Google Drive API. It allows you to easily upload, download, and delete files in your Google Drive from a Python script.

In this article we will cover:

Getting an API key from Google Drive
Installing PyDrive
Using PyDrive (authorization, creating files, searching for files, downloading, (un)trashing and deleting, uploading, etc.)
Using PyDrive in places without a web browser (e.g., Kaggle Kernels)
Examples of PyDrive in the real world

Feel free to skip to any section!

All the code we develop in this article will be available on my GitHub, here.

Getting Your API Key

First we need to enable the Google Drive API. To do that head over to this link, and click “Enable” (if you’ve already enabled the API for whatever reason, click “Manage”).

Then set your “User Type” as external, here, and click create. Next you’ll need to fill out the OAuth consent screen. All you need to do here is specify an application name.

After that, we are going to create OAuth credentials for our project. Click this link, and enter the following:

Application type: “Web application”
Name: Whatever you want, I’ll be using “PyDrive”
Authorized JavaScript Origins: “http://localhost:8080”
Authorized redirect URIs: “http://localhost:8080/”

Note the trailing “/” for the Authorized redirect URIs

When you’re done, you should see a screen that looks like this:

And when you’ve finished, click “Create.”

Close out of the pop-up window, and you’ll see a screen somewhat like this:

After that, click the download icon on the right hand of the screen to download your API key.

Move the downloaded key into the directory you will be coding in, and make sure it is named client_secrets.json . No problem if it isn’t, just rename it.

Code!

Installing PyDrive

Installing PyDrive is incredibly easy. I recommend you do this in a virtual environment, but whatever suits your purpose better.

Pip: pip install pydrive
Conda: conda install -c conda-forge pydrive

Authorization

Now that we’ve set up the API we can start using PyDrive locally. The first thing we need to do is authorize our account.

Running this will open a window in your web browser where you can log in to your Google account. When you run this for the first time, Google might warn you that the app is unverified. If this happens, click “Advanced” and then continue to log in.

Once you’ve successfully logged in, you can close the browser tab PyDrive opened, and you should see Authentication successful printed.

After authenticating your account, we need to create a local instance of Google Drive. We can do that with:

Creating a Simple File

Let’s create a simple file and upload it to our Drive.

Simple as that! You can repeat the same process for any sort of file you would like.

Searching For Files

Google Drive is a semantic (also called tag-based) file system. This means that it stores files not based on their location, but based on an ID. Semantic file systems allow you to have multiple files with the same name and to have one file appearing in multiple places.

Getting a file’s ID is required for doing anything with the file (downloading, editing, etc.). In PyDrive, we can find a file’s ID by searching for the file.

To search for a file, we use drive.ListFile. This will print every untrashed file in your Drive and its associated ID:

The ‘q’ attribute is a query we can specify. You can see a complete guide of how to structure your queries here. But here are the most common commands you’ll need:

Files with the title ‘hello’: title = 'hello'
Files with a title containing ‘hello’ and ‘goodbye’: title contains 'hello' and title contains 'goodbye'
File that doesn’t contain ‘hello’ in its title: not title contains 'hello'
File that has ‘hello’ in its content: fullText contains 'hello'
File in trash: trashed=true
File with the title ‘hello’ and is not in trash: title = 'hello' and trashed = false

Using these methods, let’s find the file we just created:

Downloading Files

Now that we’ve found the ID of the file we’re interested in, we can do a bunch of handy things.

For example, let’s download it.

Now it’s saved forever!

Trashing, UnTrashing, and Deleting

With our local copy of the file saved forever, let’s delete the copy on our Google Drive.

Be very careful when using file.Delete(). This happens instantly, without confirmation, and cannot be undone. Make sure you know exactly what you’re doing when you use it.

Uploading Files

Let’s upload the local copy of the file we just deleted back into Google Drive.

This will work for any type of file.

Creating a Folder

Creating a folder is as easy as creating any other file. All we have to do differently is specify the mimetype to be a folder.

For a full list of mimetypes Google Drive supports, check out this link. This lets you create Google Docs, Slides, Sheets, Drawings, and pretty much anything else you can think of!

Uploading a File into a Folder

The empty folder we just made is pretty lonely. Let’s give it a file to house.

First, we need to get the folder’s ID. We do this with the same type of search we used earlier. After that, we create a regular file and specify that it has a parent with the ID of the folder we just found.

Saving Your Credentials

If you want to use PyDrive in an environment without a web browser (e.g., Kaggle, we’ll talk more about this later), you will need to save your credentials to a file. By saving credentials to a file, you can also avoid authenticating yourself every time you want to re-run a program.

After manually authenticating yourself once (using a web browser)…

…saving your credentials is as simple as:

Now you can load these credentials any time you want with:

Kaggle Kernel

Let’s quickly cover how we would use those saved credentials in an environment like a Kaggle Kernel.

First, pip install PyDrive with:

!pip install pydrive

Now we are going to create a file with our credentials in it. You can call the file whatever, but I’m going to stick with mycreds.txt

echo 'CONTENTS OF mycreds.txt'>mycreds.txt

Replace CONTENTS OF mycreds.txt with whatever credentials were saved in mycreds.txt when you generated it locally. For example:

echo '{"access_token": "___", "client_id": "___", "client_secret": "___", "refresh_token": "___", "token_expiry": "___", "token_uri": "https://oauth2.googleapis.com/token", "user_agent": null, "revoke_uri": "https://oauth2.googleapis.com/revoke", "id_token": null, "id_token_jwt": null, "token_response": {"access_token": "___", "expires_in": ___, "refresh_token": "___", "scope": "https://www.googleapis.com/auth/drive", "token_type": "Bearer"}, "scopes": ["https://www.googleapis.com/auth/drive"], "token_info_uri": "https://oauth2.googleapis.com/tokeninfo", "invalid": false, "_class": "OAuth2Credentials", "_module": "oauth2client.client"}'>mycreds.txt

Make sure not to give anyone else access to this!

You can repeat the same process for creating client_secrets.json.

You can check that mycreds.txt and client_secrets.json have been created successfully with:

!ls

After that, you can authenticate yourself just like before:

Example

In this example, we will create a custom callback for a Keras model that will save it to a specified location in our Google Drive.

This class has plenty of room to be improved, but it’s certainly a starting point for whatever you’ll want to do with PyDrive!

Note: It can take a fair amount of time to upload larger files into Drive, so you probably won’t want to do this every epoch. That’s why I included the frequency argument in case you want to directly use this code.

Conclusion

PyDrive is a simple yet powerful way for integrating Google Drive into your applications. It opens a lot of doors for all sorts of cool projects — what will you make?