How to Test Jupyter Notebooks

A guide on testing Jupyter notebooks.

Jupyter notebooks are a great tool for exploratory work in science and engineering, but they are notoriously fragile. So much depends on the order of execution of the cells, it’s really easy to break your notebook completely. Believe me, it has happened to me many times. Having automated tests for your important notebooks is therefore highly recommended, as it is for any software.

There are mainly three scenarios, you usually want to consider:

• testing the values of variables in your notebook
• testing functions or classes that are defined in your notebook
• testing that the whole notebook runs without errors

You just have to install the package ipynb which allows importing notebooks in regular Python files, and you are ready to go:

pip install ipynb

An Example Notebook

As an example, we will test the following simple Jupyter notebook qm_harmonic.ipynb, which solves the stationary Schrödinger equation of the quantum harmonic oscillator.

Testing the Jupyter notebook

Here is how you can test all three scenarios mentioned above. First, you import the notebook in the setUpClass method. This loads (and executes) the notebook and makes it available for all the tests in the class. Note that in the call to the import_module function, you attach the name of your Jupyter notebook file to the string "ipynb.fs.full.”. Once loaded, you can then access all the variables and functions from within your code. Here is an implementation with the unittest framework:

The first test function tests for the variable energies defined in the notebook and asserts that it has the appropriate value for the quantum harmonic oscillator ground state energy.

The second test function makes use of the function kinetic defined in the notebook, so you can use all the tools of Python testing that you are used from regular Python testing.

And finally, there is no real need to cover the third mentioned scenario explicitly, because this is already covered by the other scenarios as well. If loading (and executing) the notebook in the setUpClass method fails (because there is an exception executing the notebook), all related tests fail automatically.

Conclusion

Testing is crucial for software quality, even more so for something so fragile as a Jupyter notebook. Nevertheless, many Jupyter users neglect that. But it can and should be done. With the steps described above, it should be easy to boost the robustness of your important notebooks.

Thanks for reading!

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Check out our Community Discord and join our Talent Collective.