Testing Polyglot Flows With Enqueuer

It hurts to say but reactivity is not an easy world, fellows.

Even not being an easy world, the benefits from reactive architecture are worthwhile.

Awesome, life is beautiful again! Take it easy, padawan.

In order to reach the reactivity’s holy grail there’s only one way, one truth and one queue. Ok, I’m lying, it doesn’t have to be one queue and there isn't a single truth. But there is no other way, as the reactive manifesto claims, there must be asynchronous messages:

Message Driven: Reactive Systems rely on asynchronous message-passing to establish a boundary between components that ensures loose coupling, isolation and location transparency

But additional complexity comes with this asynchronous approach. It's not easy to detect, avoid and handle stuff like infinite loops, dead locks and race conditions.

Polyglot flows

Depending on the system constrains an IPC protocol may fits better than others. Sometimes, there are premises that exclude some possibilities and limit the options such as network latency, hardware configuration, synchronicity and number or actors involved. Therefore it’s not unusual to have more than one IPC protocol, or even several ones, in the same flow with a set of distinct systems and different behavior. This flow, the one with several IPC protocols, is also known, at least by me, as polyglot flow.

With all of this being said, new problems arise: how can we test these polyglot flows and their several IPC protocols, asynchronous property and flowing messages? How can we keep track of all of this? — It feels like we are constantly creating things that are harder to test.

Lack of documentation

It goes even further, haven’t you ever felt like you no longer understand all the contracts and the payloads that flow by? Be honest, don’t you feel like continually documenting every contract in a wiki page kinda sucks? They’re out of date in no time.

I wish we have something that provides documentation of the messages flow and, at the same time, some CI tests over it, so it would have to be updated all the time. Something so easy to be read that anyone interested in learn how a workflow works, even, and specially, non coders fellas, could take a look at scenario file and have basic understanding of it

Take a breath, it does not have to be that hard. You are not alone.

Mock! Mock them all!

One way to test it is to mock everything! Inside your reactive system project you can write unit tests and component tests, mocking them all and pretending that the message is really flowing back and forth from/to where it should come/go.

At some point, you have to increase your test scope. At some point, pretending is not enough, you have to make sure that the message is properly flowing by. You have to check whether the configuration is properly set and you have to check the result in a database, API, file system, broker or something similar.

Furthermore, to write these kind of tests, you got to have some programming skills, what may move away non-coders from testing it. If we think this through, anyone who understands how the flow works should be able to test it.

Give me tools!

Integrating different systems, sources/destinations and formats is indispensable. Period. We infer that we need to take this seriously and use tools to do this task for us whenever possible to minimize human errors.

Some say that one of the reasons for the fall of SOA (Service-Oriented Architecture) was exactly the lack of mature tools. But it was some years ago, a lot of time has passed and the community has learned from its previous mistake, hasn't it?

Nowadays, we can find several awesome tools in the community, some of them, the best known ones, deserve to be mentioned:

Postman

Postman is a really good application. There’s a good reason it was proudly announced that it's used by 5 million developers and more than 100,000 companies to access 130 million APIs every month and in every earth in 8 solar systems in 4 galaxies! It's a lot, don't you think? It doesn’t take much effort to understand why it gets such good numbers: it's really easy to use it, including by non-coders, all you have to do is to click in two or three buttons and voilà, the magic happens! It's easily added to your CI pipeline and provides you nice features like running several requisitions sequentially one after another.

But (there is always a but), despite its deep HTTP universe exploration it limits itself to it. Even though HTTP based REST is the most widely adopted IPC protocol in the microservices world, reactive systems are not about synchronous communication, as mentioned in forget HTTP in microservices, in is REST the better option?, in Data is no longer at REST, REST is the new SOAP, and in REST turns microservices back to monoliths — I hope you're convinced by now. It's exactly the opposite, such synchronous communications deteriorate the whole system because they make your system more coupled. It's naive to expect that the other systems will be running fine and that there will be an answer in an acceptable time. Therefore, Postman loses a lot of points for not being able to handle other IPC protocols.

I believe that, and this is completely personal, not strongly emphasizing its tests functionalities makes it lose some points as well.

Mountebank

This one is able to handle IPC protocols other than HTTP. To be exact it can work with: SMTP, TCP and HTTP/HTTPS. This last one receives a special focus, of course. Given that it's able to understand TCP flows, it may work with every other TCP based protocol. All you have to do is to build some layers on top of it to make it work properly. Mountebank's real power is the capacity to expose a HTTP APIs and to provide stubs and proxies to tests executions. Through imposters, it's possible to simulate HTTP responses, parse messages and use predicates and assertions to make sure that the message is correct. I have to admit it, these functionalities rock!

But (I said there was always a but), being able to handle raw TCP message is not enough, it's still up to you to understand how the intended protocol works and build it over TCP. It's a really really really hard thing to do. Take the AMQP specification as an example.

Other than that, it's not tempting for non-coders to implement how these imposters and predicates should work.

And last but not least, as it acts only under demand it has to be used ensemble with another testing tool, one that triggers the message starting the test.

Apache Camel

It handles several several several IPC protocols. Uhay! Amazing! Finally. It triggers the message, so you don't have to use it with other testing tool. Since you control it by coding, you can do whatever you want to: nest tests, combine executions, test several systems at one time…

However (you know, "however" is just a fancy but), it's not tempting for non-coders. Actually, it's not tempting for coders either. Given that you have to code you test scenario, it requires deep programming knowledge. As I said before, it's reasonable to think that everyone who understands how the flow flows should be able to test it. These tests should not (and cannot) be a responsibility exclusive to the coder. Besides that, as mentioned in here, in breaking the camel's back section:

Most of Apache Camel’s connectors are implemented in a synchronous and/or blocking fashion, leading to inefficient use of resources and limited scalability due to high contention for shared resources.

Also, fan-in/fan-out patterns tend to be either poorly supported, requiring knowing all inputs/outputs at creation; or, lowest-common-denominator — limiting throughput to the slowest producer/consumer.

We need better tools!

There is an obvious need for better tooling. What can be done given what is known for reactive architectures: asynchronous, responsive, resilient and elastic?

Let me introduce one thing to you all, folks. Ladies and gentlemen: the great, the magical, the wonder: enqueuer.

Enqueuer to the rescue!

Enqueuer \nqr\ is a polyglot flow testing tool. A CLI framework, open source, born to fulfill this need! It triggers one input and inspects how the system reacts by analyzing its outputs messages.

Show me how to do it

In order to illustrate how enqueuer works and how to use it, I will use a fictitious example with a polyglot flow.

Fictitious example with a polyglot flow

When your e-commerce TCP server is hit, you have hit a credit card processing RESTful endpoint and notify users about this through AMQP events.

Now, you have three options to test this polyglot flow:

1. Shamelessly write no test at all; (It happens more often than we would like to admit. You know it)
2. Write test for each one of these cases separately in the codebase itself, mock them all, handle new dependencies, figure out details and deal with debugging them when they fail, document them all in a wiki page and be constantly upset about forgetting to update them; or
3. Use enqueuer and have it all right out of the box.

I'm assuming you've chosen number 3, otherwise there's no need to keep reading this article. (Please, don't go away)

Given that we want to test stuff, we have to isolate the test target, be it a single component or a set of systems. By doing this, we make sure that whenever an error occurs the target is the one to be blamed, the guilty.

After isolating it, we'll trigger the event firing a hit in the exposed endpoint and evaluate if every expected output is fulfilled. The good news is that this is what enqueuer does.

Start by installing it:

$npm install enqueuer -g

Create a configuration file and a file to describe the test scenario. Like these ones:

There are a few things that deserves special attention here. Looking at the test description file we can see that enqueuer will connect to a TCP server running in localhost:23070.

There are two expected subscriptions. The first one is a http-server. So enqueuer will run a HTTP server and listen to a (POST)call in /output1 endpoint. Once this endpoint gets hit, enqueuer asserts that it got hit and response back by mocking stuff. As you can see in response: {status: 123…

The second subscription claims that there has to be a message sent to an exchange AMQP with any routing key. Even thought we can not guarantee it, and it's a good thing not being able to do so, this message is probably sent by the system under test. In the field onMessageReceived we can pass javascript code to assert something. In the example, we create a Payload test and assert a condition on it. Of course, we can make deeper analysis there. Given that it is a javascript code snippet, you can do whatever your imagination desires.

Since that this flow is about HTTP and AMQP IPC protocols, these are the ones that the description file is worried about, but enqueuer is able to handle other protocols as well: Amqp, Http, Kafka, Mqtt, Sqs, ZeroMq, Stomp, Uds, Tcp, Udp etc.

Now we are able to run it and see its report:

We can see that 10 tests were ran in just 320 milliseconds!! How impressive is this? Some of them were automatically created by enqueuer, such as enqueuerTest.yml.Ficticious flow.Requisition #0.TCP hitting.Published and some of them were created by us, such as enqueuerTest.yml.Ficticious flow.Requisition #0.AMQP Notification.Routing key. All of them are passing. Yay.

Conclusion

The most known tools may not be enough to test polyglot flows. Enqueuer is here to supply this lack.

Mix of before mentioned tools

As incredible as it sounds enqueuer is able to:

1. to be used by non-coders. Admit it. Go ahead and take another look at it. And admire how pretty these files are;
2. to assert that whenever one thing happens other things happen in a suitable time and format. This is where the reactivity lies on. Have I mentioned that you can write your own assertions — in a damm pretty awesome way — on these flowing messages?
3. to test isolated application and whole systems at one time;
4. to trigger events by itself. Since it acts reactively and proactively no other tool is needed;
5. to mock responses. The way you want to. Programmatically, if, and only if, you want to;
6. to chain/nest events. You can reuse previous values storing them dynamically in later tests;
7. to natively support several well known IPC protocols. To name a few: AMQP, MQTT, HTTP/S, SQS, 0MQ, Kafka, TCP, UDP etc; and
8. to be easily added to a CI pipeline. Once it is ran, it returns a code telling whether any test has fail (code: 1) or not (code: 0).

You might like as well: