Tests: Which, How, Who?

Back in the days the waterfall method was still used for everything, testing was the last phase, which due to a strict deadline was often cut back on. With more agile methods nowadays, testing is a daily part of software development. As a software developer in a relative small company, we don’t have a separate testing team (nor would I want any), and it is up to us, the developers to write and perform tests.

I like the idea of Test Driven Development, but for some reason I just can not seem to push myself to write tests first. Nevertheless, I do write tests after I finished the code before committing anything. Where I work, it’s not always that obvious to write tests, a good testing culture is missing, and if we write tests it are often end-to-end tests. This even lead to entire frameworks being build that allows us to create a test by adding the input and output file. Because of my newfound love for unit testing, I highly reject this way of testing, but I often do wonder about which tests I should write. This is what I came up with as my set of best practices.

Tests can be divided into categories based on different aspects. The first one is pure hierarchical:

  • Unit Tests: The subject of my previous blog post. Only test a single unit, work on the code they are testing directly and cut all connection with other components.
  • Integration Tests: Aim to verify the connections between different components by testing them together.
  • End-to-End Tests: Check the system as a whole is working properly.

It is often tempting to write end-to-end tests, after all we are only interested in a working product as a whole. But these tests are slower, less reliable and are less clear when they fail. As a rule of thumb, you should always aim to create a unit tests, not only because it is the smallest one, but for all the reasons mentioned in the previous blog post.

Unit tests alone do not suffice, a small set of end-to-end tests is required. However, the target of the end-to-end tests is not to verify the system works, but to prove that the user his requirements are met. In my opinion should all end-to-end tests be acceptance tests, that check only the ‘happy path’ of the user. All other paths should be covered by unit tests. I have not yet discovered a purpose for deliberate writing of integration tests if a unit test is possible.

Tests can also be divided into two groups bases on the knowledge of the system. Either you know the code and you use the internal working to determine useful test cases (white box) or you only look at it from the outside (black box). As a developer I see it a valuable asset that I know how the code looks like, it allows me to write accurate and precise tests that cover the code to the fullest. But not all tests should be written in such a way:

  • Unit Tests: As they test the correct working of an internal unit, white box testing allows you to avoid obsolete test cases.
  • Acceptance Tests: Since they only cover the ‘happy path’ knowing the internal workings does not provide any benefit. Moreover, since these tests are verifying the user his requirements it is important to look at the system as a user does, without any knowledge of the internal system.

Things are falling into place, but one major question remains: “Who will write the tests?”. Bigger companies sometimes have a separate testing team, that manually check the software. I have no experience working in such an environment and I consider this to be an old-fashioned way of testing that should be replaced with automated tests. Again there is a difference between the two types of tests:

  • Unit Tests: Verifying the internal working of a component should be done by the developer who wrote it. He can use his knowledge about the unit to write the appropriate tests.
  • Acceptance Tests: Preferably they should be written by the users, but since this is highly unlikely a translation from user requirement to test should be made. This translation, if possible, should not be done by the developer but a business user. To enable a business user to create such a test (who often has no programming knowledge) an adequate framework has to be used that enables a smooth and straight forward translation. I have introduced one of such frameworks in this blog post.

Finally I should note that this blog post only covered functional tests. Tests such as performance tests are not covered, as they are a whole category on their own. They are completely unrelated to functional tests, but are just as important.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.