Can You Use Google Colab Free Version for Professional Work?
Simple discussion about Google Colab limitations in real life projects
Google Collaboratory is a Cloud Service provided by Google that allows you to use a “Jupyter Notebook-like” environment to run Python, allows access to GPU and TPU processors, and has already installed the main libraries you will need for Data Science. All this is available for free but within certain limits. The big question here is if this free service is enough for professional work, or if it is only useful for learning purposes or pilot projects with fewer data.
Let’s try to answer this question!
I have used Google Colab for two very different professional projects. These projects required different resources, and I can say that I experienced a lot of “Session Crashes” because I reached usage limits. That being said, I could complete the work only in one case, and yet I have to manage and make many changes in my initial code and analysis methods in order to “fit in the limits”.
Case 1. Job done
Job description
In this job, I had to access remote data in an SQL database hosted by Google. To access the data, I used Google Big Query integration with Python in Google Colab. The dataset is what we can for sure call Big Data, so every query needs to be optimized to run as fast as possible and retrieve only the necessary information.
Colab experience
While working via Colab I experience some database access limitations, and some queries take a really long time to process. This may be because other people were trying to access the database at the same time rather than Colab or BigQuery response. Only once did I have to change my query because Colab could not handle it because of storage limits. And these were the only problems I experienced. The final dataset used for analysis comprised only tabular data (numerical or string).
For the analysis, I built some Machine Learning models (none of these models were Deep Learning). And, at this step, I have to admit that Colab worked really well (I was not using GPU acceleration). All models were built using Scikit-Learn, and can’t remember any issues in this phase. Remember that all data stored in Colab is only temporary, so any analysis or output should be stored in your Drive or machine if you want to use it later.
Case 2. Frustration
Job description
For this project, I need to build an image classification model for medical image quality assessment. The database has about 5200 images, but offline data augmentation is needed.
Colab experience
The actual nightmare! This time, it was a completely different experience. To perform the work, I need to access GPU acceleration. However, GPU access in Google Colab is very limited, and I could access an average of 2h per day, which was clearly insufficient. Another limitation was RAM availability. It was OK to work with the original dataset, but not possible at all to work with augmented data. Long training sessions (several hours) require me to always be in front of my laptop because Colab is checking “Are you still here?” every 30 minutes, and if you are not, the session is closed. Sometimes when I tried to access GPU, the access was denied because of usage limits. This is a big problem if you are conducting extensive tests and need to experiment with different training parameters.
In conclusion, less demanding professional work can be done using the free version of Google Collaboratory, as long as the analyst adjusts the code/analysis parameters to stay within the limits. More demanding work will require higher memory, more access to GPU, or higher running times that are only available with the PRO versions. Google Collaboratory is an amazing service, with all the hard parts of setting up your laptop for Machine Learning already done. Using conda install something and pip install something will be a rare event. My only complaint about Google Collaboratory is that Pro versions are not available in my country (Portugal), otherwise, I would happily pay for it.
Thank you for reading! Don’t forget to subscribe to receive notifications about my future publications.
If: you liked this article, don’t forget to follow me and thus receive all updates about new publications.
Else If: you want to read more on the topic, you can buy my book “Data-Driven Decisions: A Practical Introduction to Machine Learning” which will give you all the information you need to start with Machine Learning. It will cost you only a coffee, and give me a small tip!
Else: Thank you!
