Improving Selenium Performance

A test should be designed such that it can be executed as fast as possible, getting the result as soon as possible, preventing interrupting the process of developing. A popular test framework to test web UI’s is Selenium. However, these tests can take much longer than they should be, which is something I experienced lately. To solve this I have investigated the cause of these slow tests, and how they can be improved.

A major reason for slow tests, are failing tests. While this may not be too common, as a failing test should be a warning and a trigger to fix what was broken, it does happen. A failing test often is waiting for something to become visible, present, selected or another state. Due to some changes in UI or bad selectors, we may be looking at the wrong element, or an event was sent wrongly.
The wait often involves a numerous amount of seconds to give the UI some time to refresh and update.

A first step to decrease the build time of a Selenium testsuite is to make sure all tests pass. This should be evident, and is not further discussed here. A second step to decrease the build time of any testsuite is to make sure your tests don’t do obsolete work. This can be any type of actions, such as restoring a backup but never using it.

While these two actions hold for any type of test, there is a third action that only holds for Selenium. The expression used to select an element, since selenium offers various ways to select an element, it is hard to know what the best approach is. Which is why I have compared them.

Each of the expression has gone through 5 sets of 100 repetitions to determine the min, average and max run-time of those 100 repetitions. The expressions are designed to locate the first meta field label in the filter box on a specific page, but this doesn’t really matter. Since some expressions are a bit long, I have decided to summarize them here, and later on reference to them by their number:

  1. getDriver().findElement(By.className(“xbl-sld-metafilter”)).findElement(By.tagName(“label”))
  2. getDriver().findElement(By.xpath(“//*[contains(@class, ‘xbl-sld-metafilter’)]//label”))
  3. getDriver().findElement(By.xpath(“/html/body/div[@class = ‘id-app-wrapper’]/div[@class = ‘id-app-container’]/div/form/section/div/div[@class = ‘id-app-scroller’]/div/div[contains(@class, ‘xbl-sld-metafilter’)]/div/div/section/div/div/div/div/span/label”))
  4. getDriver().findElement(By.xpath(“//div[contains(@class, ‘xbl-sld-metafilter’)]//section//label”))
  5. getDriver().findElement(By.xpath(“/html/body/div[@class = ‘id-app-wrapper’]/div[@class = ‘id-app-container’]//div[contains(@class, ‘xbl-sld-metafilter’)]//section//label”))
  6. final WebElement el = getDriver().findElement(By.xpath(“/html/body/div[@class = ‘id-app-wrapper’]/div[@class = ‘id-app-container’]”));
    el.findElement(By.className(“xbl-sld-metafilter”)).findElement(By.tagName(“label”))
  7. el.findElement(By.className(“xbl-sld-metafilter”)).findElement(By.tagName(“label”))
  8. el.findElement(By.xpath(“.//div[contains(@class, ‘xbl-sld-metafilter’)]//section//label”))
  9. el.findElement(By.xpath(“.//section//label”))
  10. final String xPath = “/html/body/div[@class = ‘id-app-wrapper’]/div[@class = ‘id-app-container’]”;
    el.findElement(By.xpath(xPath + “//section//label”)):

Note: the difference between 6, 7, 8, 9 and 10 is that for 6 getting the WebElement el is done every 100 times, while in 7, 8, 9 and 10 it is only done once for the run.

The performance for each of these expressions for all different browsers is shown in the table below. The table shows the minimum, average and maximum of the 5 runs in milliseconds.

Expression
(Number)

IExplorer 11 Firefox 33 Chrome 46
Min Avg Max Min Avg Max Min Avg Max
(1) 6047 6212 6356 2595 3180 3007 1323 1390 1486
(2) 3572 3650 3868 1709 1738 1727 696 724 767
(3) 3514 3561 3613 1575 1729 1816 665 681 695
(4) 3517 3536 3565 1313 1336 1370 655 684 729
(5) 3525 3532 3540 1289 1356 1462 655 663 673
(6) 10263 10350 10451 3560 3660 3805 1827 1922 2007
(7) 6959 7031 7114 2260 2373 2526 1165 1197 1223
(8) 3570 3681 3789 1346 1438 1659 621 638 659
(9) 3551 3600 3630 1368 1395 1443 606 624 644
(10) 2966 3107 3198 1229 1272 1336 531 547 561

Despite the differences between the different browsers, we see the same results for each browser individually. Since all tests needs to run on all different browsers it is good to see that all browsers favor the same expression(s).

Using By locators is about 2 times slower than using an XPath expression, even if they express the same thing: See (1) and (2).

Trying to be smart and first getting a more specific element to search with only slows things down: See (1) and (6). If you can re-use the element, you might be able to get some advantage (but not on IE): See (1) and (7).

All browsers prefer expressions (4) or (5) as one of the fastest. Expression (9) is not included, as it is a bit of a cheater. It is a bit surprising that (3) scores worse, as this specifies the exact path to the element. The most likely reason is that the path includes checks that are obsolete. The expression “div/form/section/div” in (3) for instance turns out to be the path of the first element with children.

As it turns out, nothing beats using good old XPath expressions. While this may not be the most readable way, it is by far the fastest.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.