Testing Edge Compatibility

Windows 10 is gaining popularity and so does Edge, the new browser that comes with it. With this increasing popularity it becomes important to verify the compatibility of your software with the new platform. Recently I was asked to verify that our Software behaved correctly when using it with Microsoft Edge. While it would have been possible and easy to just do some manual checks, this would not suffice and will not guarantee future changes to be compatible with Edge.

But since we already have automated tests for other browsers, it makes sense to just add Edge as a browser on which these tests have to run. Trying some new technology can be challenging, and often problems arise. Trying to get our Selenium tests running with Edge was no exception.

The first problem encountered was the local setup. As I am still using Windows 8.1 there is no way to install Edge locally and run the tests. As in many cases a virtual machine is the solution, Microsoft is being very helpful here and offers test VMs for this purpose. Having a virtual machine with Windows 10 and Edge installed I can take two different approaches:

  1. I set up a complete development environment on the VM.
  2. Using Selenium Grid I only run the tests on the VM.

The first approach leads to a very heavy VM, from which I know out of experience will run slowly. Although I had never set up a Selenium Grid, I chose for the second approach, simply because I am aware of the time and problems the first one will cause.

Setting up the Selenium Grid was not that much effort in the end, and the new experience made it worth the effort. The first run of the automated tests didn’t go that well… They all failed. The reason for this massive failure is the limited support for operations of the initial release of the Edge webdriver. I found 2 newer versions of the webdriver, one for the fall update and another one for the insiders build. Neither of them worked with my test VM because the version of Edge was too old, but updating the test VM is impossible.

The only option I had left was installing a legit Windows 10, and updating it to a newer version. After doing this my tests started to work. Some tests still failed because the driver does not handle selecting from a drop-down menu very well. For some reason it does not trigger the ‘onchange’ events of the element when selecting a new option. The only workaround for this that I found was pressing the arrow up and down keys.

With this all the tests worked, all-in-all setting everything up with Edge did not cause too many problems, or at least not as much as I expected. The only thing that was left for me to do was integrate everything with Jenkins, our continuous integration system. After doing that and running the job for the first time, it failed. A bit surprising because everything worked with my previous setup. There seems to be a difference in using Selenium Grid and running it directly because Selenium timed out waiting for the browser to be initialized, even though the browser does successfully start.

I have tried a different approach to start up the driver, instead of using the EdgeDriver(), I used the RemoteWebDriver with edge capabilities, but with no luck. Executing the driver myself didn’t work either, nor did trying a couple of times. I don’t have a clue what the problem is, I will have to investigate it.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.