10 Reasons Why Code Coverage Matters
How Code Coverage plays a crucial role in the success and quality of a product
Code coverage means what percentage of your codebase is covered by the tests or being tested. If you have no tests, then the code coverage is zero.
This article is focused on the benefit and value aspect of code coverage, if you wish to set up code coverage then you might find the following article helpful
The following are some of the benefits of using a code coverage tool
- Find out what parts of the codebase is covered by tests and what is not
- Find out code execution paths you missed
- High code coverage points to a well-written and testable code
- You are more likely to write a unit test if you notice the coverage drop
- Enforce a culture of writing unit tests using code coverage rules
- High code coverage leads to confidence in code
- High code coverage is crucial to investors
- High code coverage matters to some potential customers
- Code coverage could lead to code refactoring
- Code coverage can verify whether tests are executed or not
It’s worth mentioning that code coverage is an indicator of what is covered by the tests and what’s not, and a 100% code coverage does not mean that your product is 100% tested or is bug-free or flawless.
1. What is tested and what is not
A code coverage tool shows which part of your codebase is tested and the components that are yet to be tested. The visibility is super helpful, especially when the codebase starts to get bigger, and the team of developers grows.
2. Missed code execution paths
When the code block to be tested becomes complex and does more than a specific thing, then it gets pretty hard to figure out how many execution paths are there, however, a code coverage tool can quickly point out to the missing branches and statements. Very helpful when making sure that the tests cover all or at least critical execution paths.
3. Testable code 📈
I am with the opinion that unit tests automatically help you write better code. If you write a giant method that is packed with conditionals and does one too many things, I assure you that you would suffer when it comes to writing tests to cover all the execution paths.
Lower code coverage is the result of not many tests, which could mean that the code is hard to test. On the other hand, higher code coverage is an indication that the code is easy to test.
A developer once told me that, ~80% of the time is being spent in writing unit tests compared to ~20% that is actually the feature.
If the code is complex, then, surely it would take much longer to test every execution path. When honoring low coupling and high cohesion principle, the result would be lowly coupled, highly cohesive blocks of code that is easy to read, understand, maintain, extend, and test.
4. More likely to add tests when coverage drop 📉
Let’s assume you finished working on a feature and created a pull request. The build pipeline notices the lack of unit tests and indicates that code coverage has dropped in your pull request. What are you going to do next?
- You are most likely going to add the unit tests
- You will most likely get asked by a responsible code reviewer why the unit tests are missing
whereas if there is no code coverage in the build pipeline, all looks good, and the code gets shipped due to lack of visibility in code coverage.
The same goes for those developers who don’t bother writing tests and think they are a waste of time, the very existence of a code coverage tool push them to write tests.
5. Code coverage Rules
A code coverage tool can also be configured with rules to enforce the habit of writing tests. The following could be set up using a code coverage Rules
- Success when the code coverage metric is met
- Warning when code coverage metrics drop X percent
- Fail the build when the code coverage metrics violate certain rules
The above could be configured in the build pipeline level in conjunction with Github (or any other code versioning system) to block pull requests that are dropping the code coverage. This is helpful in larger teams, where you want the continuous integration system to police code coverage and ensure that every pull request comes with necessary tests.
6. Confidence in code when upgrading libraries
When upgrading old libraries, having a solid code coverage is a huge confidence boost and assuring that after the upgrade, the behavior of the tested code remains the same, and any issue will be flagged up before we ship the code to production.
When I have attempted to upgrade some critical dependencies i.e., Dropwizard in the past, the tests were useful in helping us resolve any issues beforehand.
All in all, higher code coverage is a boost of confidence when it comes to updating such critical libraries or making a risky refactoring that might break existing functionalities.
7. Potential investors
Every startup or company requires some sort of investment to keep going, and when it comes to investors, they are not going to put their money on a product blindly.
Based on my experience, depending on the volume of investment, the investors hire an auditing company to audit the product from a variety of perspectives i.e., security vulnerabilities, tests, code coverage, performance, technology stack, etc.
So, higher code coverage would point to a better-tested product, and it’s a good sign during potential investments.
8. Perspective customers
Depending on the type of product and service you are offering, there are customer’s who ask the one or more of the following before signing up
- Your technology stack and versions of libraries
- Where everything is hosted i.e., Amazon, Azure, etc.
- Security audit of your codebase
- Performance baseline of product
- Code coverage
A higher code coverage meaning a well-tested product would definitely stand out compared to the alternatives.
9. Code coverage could lead to code refactoring
Realistically when the product is in its early weeks and months, the objective is to get something into the market, so less attention is given to
- Testing
- Making sure libraries are up to date
- Refactoring
- Quality
Once the product is proven to have some business value and start to generate revenue, that’s when a team of dedicated developers begins to maintain, test, and extend it.
A code coverage tool could point out to areas of codebase that are good candidates for refactoring from the time went the first features of the product were designed. Here are some of the scenarios
- We have no code coverage in these areas of product because the code is tightly coupled
- We have lots of missed branches in this class, and that’s because it’s hard to test it
- We have used static classes here and there, and we cannot unit test or mock them easily
- These are unused features and blocks of code that we could remove and would increase our code coverage
The very existence of code coverage could lead to refactoring here and there, and that’s critical for the health of the product throughout its lifetime.
10. Verify whether tests are run or not
Without a code coverage tool, it is hard to know if the tests are being executed at all and whether you have covered all the code or not.
Once upon a time, we found out that specific tests were not being run, and that was due to how the tests were named.
A while ago, I created a pull request that migrates a smaller chunk of our JUnit 4 tests to JUnit 5. I ran the tests locally as well as in the Jenkins, and all looked good. During the code review, we found out that there was a -0.15% drop in code coverage. Why? After some digging, we found out that none of the JUnit 5 tests run.
Conclusion
Thank you for reading this article. I have published an article on how to contribute to open-source if that’s a topic that interests you
It’s never too late to develop habits and know things, and here is an article that I published about things I wish I knew earlier in my career
If you are a Java Programmer and coding standard is something that interests you, then you might find the patterns in the following article useful
Feel free to check out my previous articles https://medium.com/@rhamedy and follow me for future such reads. Please feel free to connect with me on LinkedIn.
