Succeeding in Data Science: 4 Essential Skills You Didn’t Learn in University

How to bridge the gap between academia and employment in the data science field

I’ve had multiple positions as a data scientist in different organizations and companies since graduating from a Dutch university in the data science field. What has surprised me and some other graduates is the gap between the skills you are taught at university and the skills required to be a valuable data science employee. In this article, I’d like to highlight the four biggest gaps in skills taught at uni and skills I’ve found necessary to perform well as a data science employee.

I’ve included useful resources in each section to help you enhance your skills in that subject area, should you wish to improve your proficiency.

Building solid data science projects

At university it is often common to work ‘Notebook-style’, using Jupyter Notebooks. The most solid project structure I set up at uni was calling different modules from a main.py file and using a requirements.txt. But using a setup.py? Or a pyproject.toml? Nope.

The requirements companies want from a data science project are much different. Your data science project should often be built as an installable package, e.g. through pip install <package_name>. A requirements.txt is sometimes necessary, but at least the package’s dependencies should be specified in the pyproject.toml (nowadays preferred over setup.py, see PEP-518).

Next to being installable, your data science project should often be able to run as a Docker container-based API in the cloud. By calling your API using REST requests, data can be processed through pipelines or even predicted with a Machine Learning model.

If you recognize that you’re also mostly doing data science projects ‘Notebook-style’, or that you are not familiar with how to build an installable Python project, I’d recommend reading e.g.

If you’re able to build an installable package, but you’re not sure how to build and run your data science project as a Docker-based API, read:

Working in a cloud-based environment

Nowadays, virtually every company and organization uses at least some cloud-based software. Currently, the most popular ones are Microsoft Azure, GCP and AWS. At my university, I hardly used any of the resources available on these cloud platforms. However, in working life, knowledge of cloud platforms by data scientists are almost as essential as their programming skills. An example of (cloud) engineering tasks I had to do as a data scientist so far:

• Deploy Python applications in a Docker container
• Build CI/CD pipelines
• Host Python packages as artifacts in the Cloud
• Build data pipelines

To be able to have some cloud knowledge of one of the platforms above is already a huge plus when applying for a data science position. Each platform has its own tracks and certificates to learn and prove your knowledge on the cloud. Each platform has different tracks for different roles. Some relevant tracks for a data scientist for each platform are:

• Azure DP-100
• GCP Machine Learning Engineer
• AWS Data Analytics

Getting such a certificate will definitely make you more valuable as a (potential) data science employee.

Working with other data scientists

One of the most important differences between data science at a university and data science in real life is that in real life, you have to work together, continuously, while in university there are often only a few team-based projects over the course of an entire bachelor's or master's.

So far, every company and organization I worked for worked either scrum or DevOps. I’ve not had a single job interview where knowledge and/or experience in one of these methods wasn’t a plus. Besides working scrum or DevOps, code reviewing is a critical task while working as a data scientist. This means you both have to be able to write clear and readable code for others to review, as well as be able to quickly and critically judge someone’s code. You can find a very interesting article about how to give constructive feedback for a code review in the link below:

Also, you can’t get away (any longer) with unclear functions or variable names, or objects which don’t adhere to Python’s PEP-8 naming conventions. If you want to learn more about how to write code that is of high quality see this article below:

Working with real-life data

At uni, most of the datasets students encounter are already heavily preprocessed. Often those datasets’ columns have meaningful names, wrong entries have been deleted and the datatypes are already rightly configured. The data you’ll encounter while working for a company or organization is regularly not even close to the standards of the datasets you encounter at uni, unless you work at a proper data-driven (tech) company.

A few examples of messy real-life data I’ve come across while working:

• Crime data of which some of the crimes’ date of perpetration was in the future.
• Column names without any naming convention [ name, Address, JobDescription, place_of_birth]
• Duplicate columns with different values [ job_title JobTitle ]
• DateTime values of different timezones without timezone implication.

Due to that real-life data is much messier than data prepared for students, expect that as a data scientist, you’ll spend most of your time talking to the business to ask for explanations of the columns and values, data cleaning, and combining data from different sources rather than performing data analysis or building machine learning models.

For more on data cleaning see e.g.:

If you want to practice on some real-life (messy) datasets, here are ten datasets to practice your data-cleaning skills on!

To Conclude

The skills that universities teach for data science may not be sufficient for excelling as a data science employee. As a data scientist who has worked in multiple organizations, I have noticed four major gaps in skills that require addressing:

• building solid data science projects.
• working in a cloud-based environment.
• collaborating with other data scientists.
• working with real-life data.

Using the resources mentioned in this article you can help mitigate the possible lack of knowledge or skills in these highly-valued areas!

Of course, each university and each study has its own data science curriculum, and some universities might have been better or worse in preparing you for working as a data science employee. Also, it has been several years since I graduated from university, so perhaps universities’ data science curriculums have improved since then on e.g. the cloud area by adding (more) cloud-related courses.

Resources

Solid Python projects https://packaging.python.org/en/latest/tutorials/packaging-projects/ https://medium.com/r/url=https%3A%2F%2Ftowardsdatascience.com%2Fhow-to-convert-your-python-project-into-a-package-installable-through-pip-a2b36e8ace10 https://readmedium.com/how-to-start-any-professional-python-package-project-9f66538ebc2 https://towardsdatascience.com/deploy-a-machine-learning-model-as-an-api-on-aws-43e92d08d05b

Cloud-based environment https://learn.microsoft.com/en-us/certifications/azure-data-scientist/ https://cloud.google.com/learn/certification/machine-learning-engineer https://aws.amazon.com/certification/certified-data-analytics-specialty/

Working with other data scientists https://www.scrum.org/resources/what-scrum-module https://en.wikipedia.org/wiki/DevOps https://mtlynch.io/human-code-reviews-1/ https://peps.python.org/pep-0008/ https://readmedium.com/6-python-best-practices-that-distinguish-seniors-from-juniors-84199d4cac3c

Working with real-life data https://towardsdatascience.com/the-ultimate-guide-to-data-cleaning-3969843991d4 https://analyticsindiamag.com/10-datasets-for-data-cleaning-practice-for-beginners/