Testing your app’s data layer

There are multiple techniques, and the ones you use depend on your testing strategy.

The techniques to test the data layer can be direct, like testing against an in-memory database (e.g. H2) or against a real database (e.g. using Docker) or indirect, like launching the backend application or testing as the end-user (UI tests). Let’s go over the direct options. The presented techniques can be applied regardless of the database technology (e.g. PostgreSQL, Redis, MongoDB).

📝 When deciding the techniques you’ll use, bear in mind the testing automation goals and where it fits in your automated testing strategy.

Testing against an in-memory database

In order to unit test the data layer, we need to properly isolate it. Let’s imagine we’ve used the repository pattern. A repository’s Dependent-on component (DoC) is a database; the plan is to replace it with an in-memory database.

The tests will call methods of the repository under test (control point). Then, we can assert by calling other SUT methods (observation point). Here’s an example of storing a user (save(user)) and then asserting it with findAll().

The main benefit of this technique is that we don’t need any extra bolts and pieces because the database is managed solely programmatically.

⚠️ Unit tests should be made only against your repositories. Don’t test implementation details like database schemas, (de)serializers, queries, and similar details. To enforce it, make them private (within their repositories).

Testing against a real database

As you can see in the diagram, this is just a variation of the in-memory database technique. In this case, we’ll use an instance of the real database; it gives us more confidence because it’s closer to the production environment. This can pay off when the production database can have slight differences from its memory counterpart. Therefore, it strengthens our testing safety net.

To make it work, you need to make sure a real database is running before these tests start. We’ll launch MySQL using Docker run command:

mysqld

# CREATE DATABASE clean_demo;
# CREATE USER 'clean_demo'@'localhost' IDENTIFIED BY 'my-secret-pw';
# GRANT ALL PRIVILEGES ON clean_demo.* TO 'clean_demo'@'localhost';
# FLUSH PRIVILEGES;

Finally, we just use those database connection details in the test setup:

My recommendation is to put this in a script (test.sh) that you can run locally and in your CI/CD so that you experience locally what happens in the CI/CD as closely as possible. Here’s an example of a Gradle project:

#!/bin/bash

./gradlew clean test

Testcontainers

Testcontainers is an awesome tool that brings together the advantages of the two previous approaches. As long as you have Docker running, you can run any kind of database in a container. The upside of this approach is that the container's lifecycle is controlled programmatically.

🛑 Mocking the database driver

Mocking the database client/driver is not a proper technique to unit test the repositories because you’d be mocking what you don’t own — in this case, the database client library — creating a dependency on code that you don’t own. You’d end up with complicated mocks of classes that you don’t control.

There are also people who created “void drivers” libraries so you don’t have to create mocks on your own. I don’t see much value in doing it, compared to testing with in-memory databases. You’d be using a driver that’s not used in real life, hence not reproducing any realistic scenario. The testing safety net would be barely changed.