Acceptance Testing Framework

A colleague of mine presented us something he saw at Devoxx this year. Gwen, the framework created by Shazam to handle acceptance testing. The idea of such a framework, of which there are many others, is that acceptance tests should not be produced by the developers but by the users. To enable them to do this, we have to provide a very easy way for them to specify them.

When I first saw the minimalism, the structure, the logic of a test written with Gwen I was amazed. It abstracted every detail and only left the essence behind. Reading such a test becomes trivial, in contrary to some of the tests I’ve seen. Though I was a lot more skeptic about another statement they made: it would allow someone to create tests faster. Though it may look like obvious this is true, the simple tests are just a facade. Beneath it lies a vast framework to represent every logical element in the application. Therefore I believe the gains are the same as if we would write good re-useable test code, but I do believe that such a framework drives us to write such code, it forces us to do so even.

In my enthusiasm I decided to experiment with this Gwen framework, but I was disappointed to see the contents of the framework. Just a couple of empty interfaces and hollow templates is what I found. The promises of nice, readable functional tests is everything but evident when starting from Gwen, and therefore I ended my experiment and begin a new quest. I was determined to produce a framework which embodied the same idea as Gwen, but had more body, and easy to use.

And as my work began I thought of a few requirements for the framework based on the problems I have encountered with tests before.

  1. Simplicity: The framework should be easy to extend and use.
  2. Clean-up: To prevent tests from corrupting the following tests, they should clean up after themselves, leaving the system in the same state as they started.
  3. Logging: In case something does go wrong, it should be easy to follow the test and see where it went wrong.

I tried many combinations, re-designed the framework many times because it didn’t match the requirements but eventually I came up with a basic first version of my own framework… Gwet (the code is publicly available on GitHub). The framework is build on top of jUnit, which I use for all my tests as I feel that jUnit is much more than just a framework for unit testing.

Where Gwen stands for “Given, When, Then”, Gwet stands for “Given, When, Executes, Then”. The first difference is already clear, I separated the when (specifying the actor) from the execution (actually doing something). There are two reasons for this:

  • Readability: I feel that separating them allows me to read the more story top-down.
  • Sequencing: It is now trivial to have a sequence of executions.

The biggest difference is the restricted freedom. I did not create a framework of just idea’s. I embedded how these objects interact through a set of methods that link together. The restrictions will enforce good use of the framework and minimize the chances of people trying to cut corners.

Another nice feature introduced by the framework is that actions (executes) and settings (given) should have an undo method implemented. This allows the framework to clean up after every test to avoid collisions with the subsequent tests that are about to run. In the end I also introduced automatic logging up to the possibilities of the framework, but it is designed in such a way that if you extend the framework you are encouraged to add all the information the framework needs to do logging for you.

3 thoughts on “Acceptance Testing Framework

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.