Automated Unit Testing

Software testing has been there since the inception of the industry, but it has been evolving continuously with the number of standards introduced by big tech companies across the industry. Software has always been tested, in the past mainly manually then with recent adoption and open source libraries developers get massive help to automate and make it part of the codebase through continuous integration. Typically the de facto standard is, a Pull Request/Merge Request/Change List (PR/MR/CL) should always have an associated test and documentation if applicable.

There are many types of testing. I still have many types in my mind from my Software Engineering course back in college, I would focus on one most important type known as Unit Test that Software Engineers mostly do..

Benefits of Testing

Testing always has benefits, and automated testing makes software development cycle faster, easier and maintainable. Some important ones are:

Quick feedback

Who would not like to know if their newly added functionality is working or not? A quick feedback coming right away from your first user i.e. unit test helps the developer to find out issues and mistakes. We will see unit tests in the later section.

Satisfaction and confidence

Especially with new Engineers, psychological safety is important, testing gives satisfaction and confidence when changes are requested. Contributors are encouraged to contribute from different teams as well.

Documentation

Tests can be treated as documentation as well for users of that API/Library/Package. They can see how to use certain functions by just looking at the Test Case.

Catches Design Issues

Functions that are hard to test can catch design issues like coupled code, unit test can help in decoupling the code and dependencies and make it testable.

Automated Testing should be part of Engineering culture across the company. Read more about Culture here.

Typically, functions are tested manually on a local machine by setting up some environment, unit test is the automated way of testing with complete isolation of functions avoiding side effects and mimicking the production behaviour as closely as possible.

Startups should encourage developers to write unit tests from day one to atleast some extent, having 100% coverage could be challenging.

In the testing world, DAMP (Descriptive and Meaningful Phrases) principle is recommended over DRY (Don't Repeat Yourself) because the goal is to keep tests easy to understand. Some of the best practices are great examples of DAMP vs DRY.

Best Practices

Test behaviours not methods

Test functions should be named based on behaviour being tested not the actual name of the function, the goal is to test behaviour not methods. E.g., a function named `division` should have multiple test cases with names like test_zero_division, test_float_division.

Keep logic aside

Tests should not include any logic including if/else and loops, this makes the test complicated to understand and kills the basic purpose.

Keep them isolated

Tests are supposed to focus on one thing, testing the behaviour of one function in an isolated environment, meaning it should not be affected by any other changes and should avoid dependencies as much as possible.

Avoid shared setup and value

Shared setup and value are pretty common might make the code look clean e.g. a shared variable is initialized in the test class and used across multiple test cases, but this makes it hard to read and follow and thus breaking the isolated environment. Shared setup actually breaks the DAMP.

Avoid test doubles

Test doubles make it hard to understand the usage of tests because it makes the test follow DRY, it's recommended to make tests as close to production as possible by avoiding Fake tests, mocks etc. Test Doubles can actually be achieved with a single line of code especially with new libraries out there which at the end of the day prefer DRY over DAMP.

Avoid frequent test updates

Tests are supposed to remain unaffected with code change, tests should rarely be updated only when the behaviour of the function has changed. If tests are updated frequently then there is an issue.

Test via public API

Testing the method via the public API is recommended, because that's the closest behaviour compared to the production environment. Private functions should/can not be tested directly, mimicking the production is the way to go.

Clear failure messages

A clear message with actual data being compared makes things a lot clearer when debugging tests. Major libraries now give the option to pass a message when a test fails or already has a default one.

Avoid code coverage dependency

Code coverage is popular nowadays, but relying on it is not the best idea. Code coverage checks the testing coverage line by line and shows the percentage of covered code which is not a good metric to rely on. It is better to focus on the behaviours being tested rather than the number of lines. Code coverage can be used as a supplemental tool.

Make it part of CI

All tests must be part of CI, from pre-submit to post-submit. Individually tests are quick to perform, but collectively they might be cumbersome. One of the ways is to do a selective test on every run, meaning run only what's needed for that given change.

In this article, we focused on the most common and important type of tests known as Unit Test, we talked about why automation is important, what are some major benefits and the best practices that are followed in big tech companies and how smaller companies can follow and make the most of it. However, there is more than this to testing, like end to end testing is another type which is more challenging when it comes to automation and requires more work upfront or even a dedicated team.