Fixing What Can Not Be Reproduced

As a software developer you will have to fix bugs reported by users. A good bug report contains these three essential elements:

  1. A description of the encountered bug.
  2. A clear scenario that caused the bug.
  3. The outcome the user expected.

A lot of users don’t report bug reports like this, but it still is a good idea to rewrite the bug report in a fixed way before starting to fix it. We have a way of how bug reports should be reported containing three sections:

  1. How to reproduce: containing the scenario
  2. Observed behaviour: what the result of the actions are
  3. Expected behaviour: what was actually expected

We don’t re-write bug reports to match this form, and that sometimes causes a lot of frustrations. Especially if the bug report is considered to be non-critical it is often delayed before working on it. But after a couple of months it becomes harder to reproduce the scenario due to changes in the product, which may have changed the scenario or outcome.

So how do we handle a bug that we can not reproduce?

A first step would be to ask for more information. The fact that we can not follow the procedure and get the same result means there must be some details missing. It is best to ask directed questions as the user does not know which information is useful. If he did know, he would have mentioned it at once when reporting the bug.

It may be a good idea to have fields in the bug report the user can/should fill in to avoid having to ask for more information. Some example of such information consists of version number, system information and even example files.

This new information may lead to a new scenario, or a new environment in which the scenario should take place. As time passes it will be harder for the user to remember the information causing the bug, and may not be able to answer your questions.

Despite everything it is still possible that a bug can not be reproduced, and the user does not have more information to tell you. It is easy to just mark the bug as “invalid”, but that is not the best approach. A good software developer needs to know how come the bug can not be reproduced. The existence of a bug indicates a flow in the code, in the work of the developer. No good software developer should sleep well at night knowing there are flaws in his code. The cause of the bug needs to be found, and learned from to prevent such bugs in the future by better testing.

The causes for such a bug can be divided into 4 categories:

  1. The bug is already fixed: in this case using an older version will show the bug. It is a good idea to investigate when the bug was fixed and refer to it in the bug report.
  2. The bug is only present in extremely rare cases: in this case we are still missing some information the user can not provide us. In this case the bug report should remain open, and the user should be asked to gather more information when the bug occurs again.
  3. The bug is part of a race condition: finding and solving this bug is hard, but some time should be invested into it. The best approach is to add extra debugging information to the product that allows more advanced debugging. This altered product should not be released but used against a test case that repeats itself until the bug occurs
  4. The user did something unexpected: In this case the user was able to push the system in an inconsistent or unexpected state, which caused the bug. Though there is no real bug here, it is a good idea to change the product such that a user can not put the system in such a state again.

A bug report always leads to learning new information and should be considered as a valuable asset instead of a shame. Of course it is true that minimizing the amount of bug reports is a good thing, but only if there are indeed less bugs.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.