Favoring Integration Tests Over Unit Tests to Verify Correctness

It only takes a little extra to inspire confidence in your system.

I’ve been writing unit tests since I started coding professionally. There’s a certain “zen” to it. It forces you to think about what it’s like to be another developer that has to use your code.

Even if that “other developer” is your future self.

Whenever I write a unit test, I not only want to verify correctness, but I also try to assess the class’ ergonomics–the developer experience that the class provides. I think an increasingly important aspect of writing good code is to write code that is pleasant to work with.

However, verifying overall system correctness is a trait that unit tests don’t possess.

Writing unit tests is dirt cheap. They’re impressively fast to write, execute, and verify a desired outcome. However, they’re also cheap in another sense. Unit tests are rarely enough to catch actual bugs that are likely to happen in production.

🔔 Want more articles like this? Sign up here.

Seeing something work in isolation under very controlled conditions is entirely different from seeing it work in a bigger setup, with multiple collaborating objects, different use cases, etc.

Carelessly mocking dependencies is a highway to disaster–and in my experience, especially so when mocking interactions with databases.

Take a second to analyze this unit test. Say we’re attempting to verify that saving a user through the “UserManager” works.

[Fact]
public void SaveNewUser()
{
   // Arrange
   var repository = Substitute.For<IUserRepository>();


   var user = new User();
   var sut = new UserManager(repository);
  
   // Act
   sut.SaveUser(user);
  
   // Assert
   repository.Received(1).SaveUser(Arg.Any<User>());
}

This is a transparent, white box test. The UserManager delegates the actual saving to a collaborator, the “IUserRepository” and we’re verifying that the “UserManager” calls the “SaveUser” method.

This is the sort of unit test that tests absolutely nothing. It doesn’t verify anything and it certainly doesn’t inspire confidence.

We’re forcing things to go well.

Such tests make us feel good because we hit some arbitrary code coverage target that lets the build server pass the code quality step, but really, it’s an empty promise that things work.

If your codebase has tests that are similar to what I just showed, then you might have a big problem on your plate.

Say your “users” table looks something like this.

-- Postgresql
CREATE TABLE "Users" (
   "Id" uuid NOT NULL,
   "Name" text NOT NULL,

   -- and many other columns

   CONSTRAINT "PK_Users" PRIMARY KEY ("Id")
);

Then saving a user constructed without any “Id” or “Name” should throw an exception, as opposed to letting you think everything is working correctly.

var user = new User(); // Should not be saved

This is obviously a very simple example with minimal interaction between a class and its collaborators. But, it nevertheless illustrates the pitfalls of blindly relying on unit tests to verify correctness.

🔔 Want more articles like this? Sign up here.

After many years of professional software development, whenever I find myself mocking anything, I take a second to think long and hard about whether it makes sense to keep that test as a unit test, or if it should be promoted to an integration test.

Now, contrast the previous test with this integration test.

public class UserManagerShould(DatabaseFixture fixture) : IClassFixture<DatabaseFixture>
{
  [Fact]
  public async Task SaveNewUser()
  {
     // Arrange
     UserDbContext context = new TestDbContextFactory(fixture.ConnectionString)
         .CreateDbContext(null!);
    
     await context.Database.EnsureCreatedAsync();
     await context.Users.ExecuteDeleteAsync();
  
     var repository = new EfUserRepository(context);
    
     var user = new User();
     var sut = new UserManager(repository);
  
  
     // Act
     sut.SaveUser(user);
    
     // Assert
     List<User> result = context.Users
         .AsNoTracking()
         .ToList();
    
     result.Should()
           .HaveCount(1);
  }
}

This will accurately mimic what our system will do in production. This test uses a fixture and a postgresql docker container is spun up–just for that single test.

Now, calling “SaveUser(user)” performs an actual insert to a real database, and, because the user is missing a name, an error is thrown: Npgsql.PostgresException 23502: null value in column “Name” of relation “Users” violates not-null constraint.

A test like this, which throws when something is actually wrong, tells us that we need to perform additional checks before attempting to save a user.

Failing tests prompt us to take action. In this case incorporate guard clauses in the UserManager’s SaveUser(user) method, checking whether the user is in a valid state.

An even better approach would be to not allow invalid user states from the beginning. But that’s another topic.

In summary…

Tests must inspire confidence that your system works, and not just the individual parts under controlled laboratory conditions.

🔔 Want more articles like this? Sign up here.

Mocking important class interactions in your system masks potential integration problems that arise from how classes communicate and exchange data with each other. Putting a little effort into using a real database pays dividends in the long run.

Catching integration problems at build-time is many times cheaper than catching easily-avoidable issues at run time (in production).

Let’s stay in touch!

Get notified about similar articles by signing up for the newsletter here and check out my YouTube Channel (@Nicklas Millard)

Don’t forget to connect on LinkedIn.