Decoupling tests from APIs

Changing interfaces (e.g. API web handlers, queue event handlers, web pages) is common and can impact many tests. How can we decouple them from those changes?

In high-level tests, why is it bad to use the test unit directly? When refactoring, think about the pain of updating dozens of tests. Also, consider the verbosity of making HTTP calls or resorting to CSS selectors in the tests. This has a significant negative impact on tests as documentation (one of the testing goals).

The solution is to apply the fundamental theorem of software engineering: “We can solve any problem by introducing an extra level of indirection”. We need testing clients to mediate access to our UIs and APIs. For UIs, you probably heard about the Page Object pattern, where the testing unit is a page and each method represents a user action (e.g. “open blog post”). Here’s how it looks in the tests (adapted from a real project):

@Test
fun `edit profile`() {
    Homepage(driver).navigate()
        .`accept cookies`()
        .`go to login`()
        .`login`("[email protected]", "qwerty")
        .`open profile`()
        .`set name`("john")
        .`save`()
}

I’ve described this pattern in terms of HTML, but the same pattern applies equally well to any UI technology. I’ve seen this pattern used effectively to hide the details of a Java swing UI and I’ve no doubt it’s been widely used with just about every other UI framework out there too. Page Object (Martin Fowler)

What if go further and generalize the Page Object pattern to target any kind of interface rather than just UIs? In other words, I propose to generalize the Page Object pattern to target any kind of interface (e.g. web APIs), for similar reasons. I’m talking about external-facing APIs (e.g. GRPC, web, event handlers, etc.), not internal code-based APIs.

Practically speaking, we need to abstract the tests’ entry points — and segregate them all in a layer. Let’s call it a SUT client or proxy. It contains a set of reusable functions to interact with the SUT, mimicking its real-life usage.

Being abstract is something profoundly different from being vague (…) The purpose of abstraction is not to be vague but to create a new semantic level in which one can be absolutely precise. Edsger W. Dijkstra

If the test subject is a web API, the abstraction is a web API client (not a generic one but one for your web API) where each method is an HTTP call (e.g. “update profile”). Here’s an example of a web API client to be used in your tests (in Python):

# in a test:
def test_cant_create_repeated_users():
   App().start(1234)
   api_client = ApiClient("http://localhost:1234")
   api_client.create_user('[email protected]')
  
   response = api_client.create_user('[email protected]')

   assert response.status_code == 409

# in a shared place only visible by tests:
class ApiClient:
   _client: HttpClient

   def create_user(self, name: str):
      ... # http call with _client
   def get_user(self, id: str):
      ... # http call with _client
   def list_users(self, offset: str, count: str):
      ... # http call with _client

The general recipe is to start by extracting all API invocations as reusable functions into a separate file — a shared testing utility. Function names should start with verbs that depict user actions. Name the clients with business domain names (e.g. OrderConsumer) (ubiquitous language). All tests can now make use of those utilities.

Make sure you don’t have any reference to the internals of your app (e.g. domain entities, DTOs, enums, etc.) to keep the tests black-boxed.

Here’s a more realistic example (in Kotlin):

object HttpClient {

    private val httpClient = newHttpClient()

    fun `create user`(email: String, name: String, password: String): HttpResponse<Void> =
        httpClient.send(
            newBuilder()
                .POST(ofString(""" { "email": "$email", "name": "$name", "password": "$password"} """))
                .uri(URI("http://localhost:8081/users")).build(), discarding()
        )

    fun `list users`(): HttpResponse<String> =
        httpClient.send(newBuilder().GET().uri(URI("http://localhost:8081/users")).build(), ofString())

    fun `delete user`(email: String): HttpResponse<String> =
        httpClient.send(
            newBuilder().DELETE().uri(URI("http://localhost:8081/users/$email")).build(),
            ofString()
        )
}

Check examples of the client being used.

Beware that these clients should be dumb; they should be only mirrors of the real APIs. That said, don’t add any logic or abstraction to the clients. For example:

• don’t validate the inputs — make them optional raw strings (e.g. in a paginated listing, count should be a string). This way, you can also use the clients to test for invalid inputs.
• don’t do any assertions in the clients, to avoid compromising them to happy scenarios only.
• don’t store any state in the clients; instead, store it outside and pass it in.

On the other hand, don’t try to abstract the responses. For example, in web APIs, the response is too complex to be worth the effort.

Our tests can now use clear and stable surface areas as entry points. But aren’t those entry points (interfaces) also implementation details? Ultimately, everything is an implementation detail but we need to draw the line somewhere.

These SUT clients are especially useful if you follow a vertical approach to testing, where the unit is a use case and each test focuses on business interaction (in opposition to testing low-level technical details).

I’m against shared utilities but this is an exception; besides, this is testing code, so the practices vary a bit from the implementation code. At first, the proposed abstraction seems overkill, but it pays from early on. The tests become:

• Easier to write: You end up with more help from the IDE: it autocompletes/validates the SUT client functions and parameter names. The set of clients ends up looking like a DSL that documents the app’s intents.
• Easier to read: The pattern tackles repetitive code and hides the underlying technical details. By replacing numerous complex calls, such as HTTP calls, with simple function calls, the focus shifts from the “how” to the “what”. By making “Arrange, Act, and Assert” steps all at the same level (Single Level of Abstraction), tests become more readable and demonstrate strong documentation capabilities. They read like a story and are easy to understand.
• Improved safety net: As a rule of thumb, each test should use the same entry point (e.g. a web API) to Arrange, Act, and Assert. Due to that, some things are indirectly tested in the Arrange and Assert of multiple tests so they don’t need to be directly tested. Probably, a wider range of combinations are tested.
• Less refactoring pain: SUT clients insulate the blast radius of potential changes to APIs — these can be easily made in a single SUT client, rather than having to update dozens of tests. This can protect tests from small updates, such as updating the URL of an HTTP endpoint, as well as bigger changes, such as replacing a communication protocol (e.g. from GRPC to HTTP). This approach allows for easier maintenance and updating of the tests over time.