Run Python files with Cron on Linux - Job Scheduling Guide

Learn to use cron to schedule jobs to run automatically at specific times and days on Linux systems.

A crontab file consists of lines — called cron jobs — which specify when to run what command. And so they consist of 2 parts:

• Scheduled times — This part consists of 5 parts that together determine when to run the cron job.
• Command to run — This is just a regular Linux command that will be run at the scheduled times.

How to set the scheduled times for cron jobs in a crontab file?

The scheduled time of a cron job is specified with 5 parts separated by spaces, for example: 0 7 25 12 *.

The parts have their own roles and specify:

• what minute to run (0–59)
• what hour to run (0–23)
• what day of the month to run (1–31)
• what month to run (1–12)
• what day of the week to run (0–6 where 0 is Sunday and 6 is Saturday)

The cron expression 0 7 25 12 * stands for 7:00 on the 25th of December.

Cron expression syntax

Here is an overview of some notations that you can use in the cron expression:

• * means any value
• with , you can separate multiple values — 3,6 means 3 and 6
• with - you can specify a range — 3-6 means 3, 4, 5 and 6
• with / you can specify a step size — */20 means every 20th minute, so 0, 20 and 40 — 10/20 means every 20th minute from the 10th minute, so 10, 30 and 50.

As an example, the cron expression */5 7–17 * 1,4,7,10 * means every fifth minute of the hours 7 through 17 on every day in the months January, April, July and October.

Cron day of the month and day of the week

Note that there are 2 parts of the expression that are about days. Days of the month and days of the week.

When you use the * character for only one of them, the other part is used.

For example, 0 7 * * 5 means every Friday on 7:00. And 0 7 5 * * means 7:00 on every fifth day of the month.

But when you specify a value other than * in both of them, every day that matches one of the specifications ‘counts’. For example, 0 7 1 * 5 means 7:00 on the first day of the month and every friday.

Crontab guru

With this online tool you can check whether your expression will make the command run at the times you intended.

You can put in a scheduling expression like:

0 7 * 3,6,9,12 1-5

And it will convert it to a human readable output like this:

“At 07:00 on every day-of-week from Monday through Friday in March, June, September, and December.”

How to make the command to run a Python file in a crontab file?

After specifying the scheduled time, you can place an extra space on the cron job line and then add the command to run.

The standard command to run a Python file in Linux is something like this: python3 filepath_of_script.

But when working with crontab it might be better to work with absolute filepaths for both the Python executable and the filepath of the script.

Filepath to Python in Linux

To find out where Python is located on your Linux environment you can run this command: whereis python. But that gave me this output: python:.

So I tried again with whereis python3 and that gave me more helpful output:

python3: /usr/bin/python3 /usr/lib/python3 /etc/python3 /usr/share/python3 
/usr/share/man/man1/python3.1.gz 

I will be using the filepath: usr/bin/python3.

Filepath to a Python file in Linux

If the Python script that we want to run is called daily_task.py and our command prompt is in the directory where that script is placed, we can run the command realpath daily_task.py. That command returns the absolute filepath of a file, in this case /home/be/daily_task.py.

The command to run a Python file in Linux

By combining the path to Python and the path to the Python file we get the command to put in the cron job: /usr/bin/python3 /home/be/daily_task.py.

How to create a cron job to put in crontab?

By combining the scheduling expression with the command you make a cron job that can be added as a line inside crontab. Here is an example of a cron job that runs the Python script daily_task.py everyday at 7:00:

0 7 * * * /usr/bin/python3 /home/be/daily_task.py

Capturing output and errors of jobs in crontab

You can capture the output of cron job commands in a file. That way you generate a sort of log for your cron job.

In Python the output that will be put in a file is whatever is printed by the Python file.

The way to capture output of a cron job is adding > logfile_filepath to the line in crontab.

In the example below we put the output of daily_task.py — generated by print statements — in the file log2.txt in folder /home/be:

* * * * * /usr/bin/python3 /home/be/daily_task.py > /home/be/log2.txt

By using > the output that is generated by the cronjob replaces the old file.

To add the output to the file instead of replacing it you can use >> in your cron job. Here is an example of appending to a cron job log file:

* * * * * /usr/bin/python3 /home/be/daily_task.py >> /home/be/log2.txt 

Unfortunately, sometimes an error occurs when running a Python file. To also log Python errors in the cron job log file you can add 2>&1 to the end of the cron job line in crontab. Here is an example:

* * * * * /usr/bin/python3 /home/be/daily_task.py >> /home/be/log2.txt 2>&1

Running Python files with virtual environments

When your project is inside a folder and has its dependencies in a virtual environment (venv), you have to specify the filepath of the environment’s version of Python.

For example, we could have a directory /home/be/project1 and make a virtual environment inside that folder called venv. A Python file called main.py could be placed in the project1 folder as well.

In that case, the script could be scheduled by putting this cron job in crontab:

* * * * * /home/be/project1/venv/bin/python3 /home/be/project1/main.py

How to edit and view a crontab file?

To start editing a crontab file you can run the command: crontab -e.

The first time you run that command you are asked to select an editor:

no crontab for BetterEverything - using an empty one

Select an editor.  To change later, run 'select-editor'.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.basic
  3. /usr/bin/vim.tiny
  4. /bin/ed

Choose 1-4 [1]:

For this guide we choose option 2: VIM (basic) by typing 2 and pressing enter.

VIM is a text editor inside the command line interface that might take some time getting used to. But I think that with these notes for working with VIM it is quite easy:

• After running crontab -e the crontab opens in the text editor, in our case in VIM. In VIM you can use the left, right, up and down arrows to navigate through the lines.
• If you press i you can edit the crontab. You can just type characters to add and press DELETE and BACKSPACE to remove characters. By hitting ENTER you can add new lines.
• When you are done editing you can press ESC.
• And to save the changes you can type :wq and press enter. This will also close the VIM text editor.

To view the crontab you can run the command crontab -l.

Summary

In this guide we learned:

• about the crontab syntax to determine the times to run a cron job.
• how to run Python files in Linux even those in virtual environments.
• how to capture output and errors from cron jobs in a log file.
• how to use the VIM text editor to edit the crontab file.

Thank you for reading!

I hope my post was helpful to you!

You can get full access to all my posts by joining Medium. Your membership fee directly supports me and other writers you read. You’ll also get full access to every story on Medium: