A Pragmatic Idealist’s Grumblings About JavaScript Testing

I have never in my life felt so conflicted about something. It’s worse than salted caramel. Much worse. More like Tarantulas. Fascinating yet horrifying. You feel compelled to take a macro shot of it while running for your life, all at at the same time! That’s how I feel about JavaScript testing, and I will apologise profusely upfront for sharing an idealist view on the matter, but I feel it needs to be shared, because for the benefit of every JavaScript software engineer’s mental health out there, these things need to be said out loud, because who knows, maybe someone will be inspired to improve it all. The alternative — the only alternative — is coining a new role called software developer therapist, and us plopping our oversat behinds on their couches an hour a week for the rest of our careers.

I love software testing.

I have to make that statement, otherwise I’ll come off as one of the Spartan developers out there who just writes features and walks away in slow motion with cool designer shades on, fingers crossed, hoping for the best. It’s a distinct group I both acknowledge and respect, just don’t exactly feel part of, though I always leave space for irrefutable arguments, so if you want to change my mind, go for it in the comments.

For the time being though, I am in a committed relationship with automated software testing, and every time a test of mine passes, a Christmas bell rings in my ear. Only then I feel entitled to walk away with my equally cool designer shades on and stare into a piñacolada on some remote tropical island, like a boss. Automated tests have saved me many times. Just the other day failing tests uncovered not one, not two, but three un-adressed scenarios in my code that otherwise would have gone to production and leave just about 70% of the app’s users staring at a broken UI. And when you’re dealing with millions and millions of users, 70% of them, we can all agree, is nothing to sneeze at. As you read this, do keep the scale of context in mind, as I think it matters a lot.

Summa summarum (latin), good tests give one confidence and reassurance they’re doing their due diligence to prevent user disappointment, wasting valuable company resources, and well, their own reputation as an engineer.

JavaScript software testing though…

Well, let me put it this way. Half the time it feels like a magic potion, and the current flavour of the month seems to be Jest + React-testing-library + MSW (mock service worker) + Codecept (or Cypress). There is nothing inherently wrong with this cocktail of testing libraries, except for the fact that it happens to be the current flavour of the month.

Not so long ago we — at least react developers that is — were all raving about Enzyme, and before that, about something else. Give it a year or two, Kennt C. Dodds and his React Testing Library won’t be the hottest kid on the block either, and so on, and so on.

I can totally understand the highly dynamic and evolutionary space JavaScript is in, but as a pragmatic and highly delivery-oriented person and engineer I just cannot shake off the feeling that all this slows things down and costs companies tons more than it needs to.

Writing tests can be literally painful…

On average, at least within the senior software engineering space, the consensus is that one must spend at least as much time writing tests as they did writing the feature and for the most part I subscribe to that, except not for the expected or… right reasons.

I see this all too often — engineers avoiding writing tests at all cost — and often when they do, the tests are really just there for show, especially unit and integration tests. The answer as to why that is, is far from what we’d expect. Yes, there is the group that doesn’t want to write tests, and yes, there’s also a group of just lazy developers across the industry, and probably most of us have been part of one of those groups at one point or another, let’s not pretend we haven’t. The larger group however, is the one that wants to write comprehensive tests, but writing them becomes so incredibly frustrating, it becomes the least satisfying stage in the development process.

You see, for the most part, JavaScript is more or less the same it was for years. ES6 is kind of the standard now, and we all happily code away our arrow functions, consts, lets, spreads and all that good shtuff (not a typo, look it up). React hasn’t changed that much either, I mean sure, we have hooks and context API, but it’s nothing mind-boggling really. Angular hasn’t changed all that much in its 10+ iterations either, nor did Vue or Svelte. However, when it comes to testing, the syntactic sugar we eat on a daily basis is enough to render a whale diabetic in mere hours and we all know those poor guys are already facing extinction, diabetes is the last thing they need!

Sure, I’m exaggerating, but I’m trying to highlight a point here too — testing libraries are often so opinionated and chock-full of syntactic sugar, that every time the testing cocktail is changed, it’s a real hurdle, and becomes an extremely time-consuming genuine sweat-inducing effort to test one’s code.

It should absolutely not be more difficult to write tests for a feature, than writing the feature itself.

Now let me mock mocking for a second…

I find it a little bit ironic that the same library — RTL — that wants to enforce integration testing that really emulates what the user experiences, relies so heavily on mocks. Now, of course, the good part of these is that you get to have all the control you need in your tests. The not so good part is that, in the real world, you will never have that control. You see, the TCP protocol on which a lot of the web runs — yay, I can finally use my CCNA to explain something — is extremely unreliable and has been since its inception. Add to that all the complexities of an enterprise application, the various services it relies upon, CDNs, firewalls, etc and you got yourself a system that is borderline magical, and every time it successfully runs, a lot of things have to go right. Break — even for a second — something in that chain, and you got yourself a scenario you hopefully tested for, because if not… you’re in trouble.

But that’s not all. What’s even more hilarious is that at the end of the day, one still ends up on StackOverflow talking about flaky tests, slow tests, tests that are too fast, tests that need x seconds of wait time added and so on. The same pain-points E2Es had, and RTL was supposed to fix. On which note…

Integration tests cost less than end-to-end tests…

Said someone at some point and provided no real data to support their arguments but we believed them anyway. Sure, E2Es take a tad longer to run, after all they do exactly what a user would do, clicking all the things, triggering all the services, potentially running on the infrastructure a real user would. That part of the argument is true. But… they’re much easier to write, much easier to read, debug, and guess what? They don’t require setting up a ton of mocks.

Just because a test runs longer, doesn’t necessarily make it more expensive.

Proof is in the fact that we have automated E2E libraries and frameworks. Running something automagically on a computer that spits out a report at the end, is miles cheaper than having a human spend time on it, right? So if that revelation happened at some point in the past, I wonder how is it that we end up coming up with more and more convoluted testing libraries requiring humans to spend more time writing tests than features, instead of doing the exact opposite, and create testing strategies, that are so simple to implement, that everyone will jump at the opportunity, to write really robust and comprehensive tests.

Let’s break down the math

Suppose I have 8 hours to write a tiny feature, out of which I spend 4 hours on developing it, 2 hours figuring out how to make the integration tests run in that particular scenario, write the mocks, and cover at least a couple basic test cases, I am left with merely another two hours to test edge cases — which is so not enough. Every single time I had code of mine break in production, it was an edge case that wasn’t covered. Doesn’t happen often, but when it does, it’s beyond embarrassing. The root cause? Because by the time I managed to write the base cases, I had no mental juice left for anything else, I was happy that I managed to write passing tests. That, in my book, is a false sense of security. We all know that test coverage doesn’t equate to testing all possible cases. Not by a long-shot. My happiness as a developer should come from knowing that I covered every single edge-case I could think of. That is what my 4 hours of testing should be spent on, not figuring out how to mix the test cocktail so it does what it’s supposed to and doesn’t induce literal head-ache in the process.

But testing is easy… right?

Here’s the problem with every single testing library I have seen so far. All the provided documentation is either incomplete, poorly written (writing documentation is an art in itself), or lives in a reduced context. Writing tests for a ToDo app running entirely on a local machine is so very far from the realities of many JavaScript engineers, that it genuinely makes me angry. On the surface, every testing library is a piece o’ cake to set up and use. Try and do that in an enterprise setup, in a monorepo or single-spa, and you’ll be spending more time on StackOverflow than ever before. Heck, you’ll switch between search engines too, just in case Google didn’t return all the possible answers, and Bing or DuckDuckGo might.

Could it just be me?

Let’s face it, while chances are slim, one has to ask whether they’re the only one struggling or feeling frustrated. It’s the right and honest, grown-up question to ask. To both my relief and disappointment, I am not alone. Proof of that are the thousands and thousands of questions on StackOverflow. Part of me however wants to play the devil’s advocate, and point out something rather interesting.

Among the various nuanced groups of developers and software engineers, there are two major types: the coders and the creators. Neither of these group-names came to be through any scientific method. They’re anecdotal. Some professionals might fall into both categories or migrate between the two. But I think, one aspect is certain, coders won’t find any — or very little — of my above grumblings warranted. Creators however, will likely do. Neither group is better than the other, and development needs both. But making the clear distinction between the two major personality types, I feel, is crucial.

The Coders

During the last decade I noticed that there are a lot of software engineers who get incredibly excited by actual code, syntax, syntactic sugar, refactoring, coming up with programming patterns, paradigms, going down the rabbit-hole of libraries they use, getting into philosophical discussions about languages and coding practices. I call them coders, because their passion and drive seems to exist within the realms of code. These are often the developers and engineers who will not only not mind the dynamic nature of the JavaScript space, moving from one testing library to another, never losing focus of the next interesting challenge, but downright live for it. The thrill of chasing that perfect line of code.

The Creators

On the other side — a group which I increasingly find myself in — are the creators. Creators often care very little if at all about the nitty-gritty details of a library, are a lot less prone to jump to the new shiny thing, simply because what excites them is solving product-level problems and delivering solutions users can benefit from. We all have our reasons for becoming software engineers. Mine, from the very beginning, was building and delivering things. JavaScript and Dart just so happened to become the languages I use to achieve that. But I started with C and continued with another six or seven languages. That’s why you’ll never find me in a heated argument about one language versus another. The concerns a creator will have, are more around whether that language will make it harder to build, deliver and maintain the end result, or easier. Will it delight the user and will it cost less than the alternative? That’s what keeps them up at night.

Right, so now what?

I’m a pragmatist, sharing an idealist view, so that in itself is a complicated Jekyll & Hyde kind of situation already. I expect comments to reflect the full spectrum of software engineering, because really, this is not just a JavaScript problem. In an increasingly digital world the conflict between fast delivery and high-quality products will only increase. One cannot swipe bugs under the rug, so testing will remain crucial, but I can’t but wonder, whether for the good of human-kind, my own and every engineer’s sanity, should we not — as an industry — focus a bit more on robust, long-lasting, efficient rather than experimental approaches.

Should it not be perhaps the industry’s common goal, to release well-tested products with greater ease?

After all, nobody really wins from an unclear path to success. Heck, every time I run a test suite that fails because I didn’t quite write my test correctly, a little branch of an actual tree probably dies somewhere. It’s not really talked about, but writing software has an environmental impact too, so perhaps for the good of the planet if for no other reason, we should do everything possible to reduce distractions, frustrations and blockers from our development practices. Coders can come up with super-robust and efficient tools, and creators can merrily go forth and create. Sounds both ideal and pragmatic, if you ask me. Also, keeps all of us off the therapist’s couch…

Attila Vago — Sr. Software Engineer building amazing ed-tech software. Cool nerd since forever, writer of codes and blogs. Web accessibility advocate, Lego fan, vinyl record collector. Loves craft beer!