authors are vetted experts in their fields and write on topics in which they have demonstrated experience. All of our content is peer reviewed and validated by Toptal experts in the same field.
Simon has been a technical tester for over 18 years. He started at IBM, moved to EADS, then Fujitsu, and now runs his own company.
PREVIOUSLY AT
Every year, test automation engineers from around the globe will research the latest tools and techniques in order to make their test automation framework more stable, faster, and easier to use and maintain. This is vital to ensuring the continued widespread adoption of their framework within their company. In general, bloated, out-of-date frameworks fall out of fashion quickly.
In this article, we’ll take a look at some of the ways you can update your framework for 2019 and how to be prepared for 2020. In order to improve my framework, I always concentrate on the “pain points.” These are the areas that are complex to set up or cause the most failures. I identified three main areas that I wanted to simplify or improve on:
Selenium Grid is notoriously tricky to set up and can fail without warning. I wanted to see what, if anything, had improved here. I also wanted to investigate if any new waits
had been added to the Selenium API to improve the stability of any tests I created. Finally, I wanted to see if I could start interacting with Chrome DevTools through Selenium, which have become an essential part of any tester’s toolkit.
Selenium Grid is notoriously hard to set up, unstable, and difficult to either deploy, or version control, on a CI pipeline. A much easier, stable and maintainable way is to use the pre-built Selenium Docker images.
Note: The one downside of this method is that IE (Internet Explorer) is not supported, as it’s not yet possible to containerize the Windows operating system.
To get up and running, first you need to have Docker and Docker Compose installed on your machine. If you’re running Windows 10 or a Mac, then they will both be installed through the Docker Desktop.
The official Selenium repository on Docker Hub contains pre-built Docker images for your Selenium Hub and Firefox and Chrome Nodes.
The easiest way to use these in a local Selenium Grid is to construct a Docker Compose file within the root directory of your project. Name the file docker-compose.yml
to keep things simple.
I’ve included an example below which creates the following Grid:
#docker-compose.yml
version: "3"
services:
selenium-hub:
image: selenium/hub:3.141.59-neon
container_name: selenium-hub
ports:
- "4444:4444"
chrome:
image: selenium/node-chrome:3.141.59-neon
volumes:
- /dev/shm:/dev/shm
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
firefox:
image: selenium/node-firefox:3.141.59-neon
volumes:
- /dev/shm:/dev/shm
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
The Docker Compose file describes the setup of your Grid. For more information about creating Docker Compose files, please see the official documentation.
To start your Grid, simply use any terminal window (a powershell
or cmd
window in Windows) to run the following command from the root directory of your project:
docker-compose up
You can connect to your Selenium Grid in exactly the same way as you normally do, as the Hub is listening on port 4444 of your local machine. Here’s an example where we set up our Driver to use our Chrome Node.
// Driver.java
protected static RemoteWebDriver browser;
DesiredCapabilities cap = new DesiredCapabilities();
ChromeOptions chromeOptions = new ChromeOptions();
cap.setCapability(ChromeOptions.CAPABILITY, chromeOptions);
cap.setBrowserName("chrome");
driver = new RemoteWebDriver(cap);
You can then use the TestNG library to run your tests on multiple nodes in parallel as usual.
It’s worth noting that it is possible to have multiple browsers running on each node. However this is discouraged, and using one browser per node is considered best practice for optimum performance.
If you want to see what’s happening on the browser so you can debug your tests, then it’s worth having a debug
version of your docker-compose.yml
file that downloads the debug
browser nodes. These contain a VNC server so you can watch the browser as the test runs.
It’s also possible to run the browsers headlessly for increased speed (the usual way) and Selenium also provides base
versions of the images so you can build your own images if you need additional software installed.
To create a stable version of the Grid for your CI pipeline, it’s also possible to deploy your Grid onto Kubernetes or Swarm. This ensures that any Dockers are quickly restored or replaced if they do fail.
As any test automation engineer knows, waits
are crucial to the stability of your test automation framework. They can also speed up your test by rendering any sleeps
or pauses
redundant and overcome slow network and cross-browser issues. Below are some tips to make your waits
even more resilient.
The ExpectedConditions
class has grown over time and now encompasses almost every situation imaginable. While ExpectedConditions.presenceOfElementLocated(locator)
is often enough, it’s best practice to use the methods within the ExpectedCondition
class to cover every user action by embedding them into your Actions.java
class. This will bullet-proof your tests against most cross-browser or slow website issues.
For example, if clicking on a link results in a new tab opening, then use ExpectedConditions.numberOfWindowsToBe(2)
. This will ensure that the tab is there before trying to switch to it.
You can also use a wait
to ensure that you capture all the elements present on the page when using findElements
. This can be especially useful if it takes time for a search
page to return its results. For example, this line:
List results = driver.findElements(locators.RESULTS);
It may result in an empty List
array if your search results haven’t loaded yet. Instead, it’s better to use the numberOfElementsToBeMoreThan
expected condition to wait for the results to be more than zero. For example:
WebElement searchButton = driver.findElement(locators.SEARCH_BUTTON);
searchButton.click();
new WebDriverWait(driver, 30)
.until(ExpectedConditions
.numberOfElementsToBeMoreThan(locators.RESULTS, 0));
List results = driver.findElements(locators.RESULTS);
results.get(0).click();
Now, your findElements
command will only run after the search results have been returned.
This wait
is also useful for finding single element when you’re dealing with a front end that doesn’t play nicely with Selenium (e.g., Angular websites). Creating a method like this will protect your tests, making them much more stable.
protected static WebElement waitForElement(By locator){
try {
new WebDriverWait(browser, 30)
.until(ExpectedConditions
.numberOfElementsToBeMoreThan(locator, 0));
} catch (TimeoutException e){
e.printStackTrace();
Assert.fail("Timeout: The element couldn't be found in " + WAIT + " seconds!");
} catch (Exception e){
e.printStackTrace();
Assert.fail("Something went wrong!");
}
return browser.findElement(locator);
}
It’s even possible to wait for elements to no longer be visible. This is especially useful if you’re waiting for a pop-up to disappear after you’ve clicked on the OK
or Save
button before proceeding with your test.
WebElement okButton = driver.findElement(locators.OK_BUTTON);
okButton.click();
new WebDriverWait(driver, 30)
.until(
ExpectedConditions
.invisibilityOfElementLocated(locators.POPUP_TITLE)
);
All the methods described above and more are listed in the official documentation. It’s well worth spending ten minutes reading through all the possibilities and improving the stability of your framework.
A good way to build resilience into your waits
is by using logical operators. For example, if you wanted to check that an element has been located AND that it is clickable, you would use the following code (please note that these examples return a boolean value):
wait.until(ExpectedConditions.and(
ExpectedConditions.presenceOfElementLocated(locator),
ExpectedConditions.elementToBeClickable(locator)
)
);
The OR operator would be appropriate if you weren’t sure whether or not the title of the page might change. Then you can include a check of the URL if the first condition fails, to confirm that you’re definitely on the right page.
wait.until(ExpectedConditions.or(
ExpectedConditions.titleIs(expectedTitle),
ExpectedConditions.urlToBe(expectedUrl)
)
);
Or if you wanted to ensure that a checkbox is no longer enabled after an action is performed on the page, then the NOT operator is appropriate.
wait.until(ExpectedConditions.not(
ExpectedConditions.elementToBeClickable(locator)
)
);
Using operators can make your waits
more resilient and result in tests that are less brittle.
Running your web app on localhost or on a local network can give a false impression as to its performance when running in the wild. The ability to throttle various upload and download speeds will give you a better representation as to how your application will run over the internet, where timeouts can cause actions to fail. We can start to simulate this using the power of Chrome’s DevTools.
The following code will open the Toptal home page using different download and upload speeds. First, we’ll store our speeds in a TestNG data provider using the following code:
import org.testng.annotations.DataProvider;
public class ExcelDataProvider {
@DataProvider(name = "networkConditions")
public static Object[][] networkConditions() throws Exception {
return new Object[][] {
// Upload Speed, Dowload Speed in kb/s and latency in ms.
{ 5000 , 5000, 5 },
{ 10000, 7000, 5 },
{ 15000, 9000, 5 },
{ 20000, 10000, 5 },
{ 0, 0 },
};
}
}
Note: The upload and download throttling is in kb/s
and the latency is in ms
.
Then, we can use this data to run our test under different network conditions. Within the test, the CommandExecutor
will execute the command in the browser’s current session. This in turn will activate the necessary settings in Chrome’s Developer Tools functionality to simulate our slow network. The code within the if
statement can be included in a @BeforeClass
method when running a suite of tests.
import org.testng.annotations.Test;
import com.google.common.collect.ImmutableMap;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.openqa.selenium.remote.Command;
import org.openqa.selenium.remote.CommandExecutor;
import org.openqa.selenium.remote.Response;
public class TestClass {
// load our data provider
@Test(dataProvider = "networkConditions")
public void test(int download, int upload, int latency)
throws IOException {
// only run if the network is throttled
if (download > 0 && upload > 0) {
CommandExecutor executor = driver.getCommandExecutor();
// create a hashmap of the required network conditions
Map map = new HashMap();
// you can even test 'offline' behaviour
map.put("offline", false);
map.put("latency", latency);
map.put("download_throughput", downloadThroughput);
map.put("upload_throughput", uploadThroughput);
// execute our code
Response response = executor.execute(
new Command(driver.getSessionId(),
"setNetworkConditions",
ImmutableMap.of("network_conditions", ImmutableMap.copyOf(map))));
}
// Open the website
driver.get("http://today.youfa110.com/");
// You can then check that elements are loaded etc.
// Don't forget to use waits!
}
}
Browser cookies can cause different behaviors in your application, depending on whether or not they have been saved from a previous session (e.g., the application might load with a user already logged in). It’s good practice to clear out your cookies before each test run to ensure that they don’t cause problems.
The code below allows you to delete all your cookies:
driver.manage().deleteAllCookies();
You can also delete a cookie by name:
driver.manage().deleteCookieNamed("CookieName");
Or get the contents of a cookie:
String myCookie = driver.manage().getCookieNamed("CookieName").getValue();
Or get all the cookies:
List cookies = driver.manage().getCookies();
Selenium 4 will be released over the next few months. It’s still under development, but as an alpha version has already been released, it’s worth taking a look at what improvements it will offer.
Note: You can keep track of their progress by looking at the roadmap.
No longer will Selenium need to communicate with the browser through the JSON wire protocol; instead, automated tests will communicate directly with the browser. This should address the famous flaky nature of Selenium tests, including protecting against browser upgrades. Hopefully test speed will increase also.
The Selenium Grid will be more stable and easier to set up and manage in Selenium 4. Users will no longer need to set up and start hubs and nodes separately as the grid will act as a combined node and hub. Plus, there will be better support for Docker, parallel testing will be included natively, and it will provide a more informative UI. Request tracing with Hooks will also help you to debug your grid.
The Selenium documentation will be getting a much needed overhaul, having not been updated since the release of Selenium 2.0.
Support for Opera and PhantomJS browsers will be removed. Headless running can be performed with Chrome or Firefox, and Opera is built on Chromium and therefore Chromium testing is seen as sufficient for this browser.
WebElement.getSize()
and WebElement.getLocation()
are now replaced with a single method WebElement.getRect()
. However, as these are often used to create screenshots of a single element, it’s worth knowing that there will also be an API command to capture a screenshot of an element in Selenium 4.
For WebDriver Window
, the getPosition
and getSize
methods will be replaced by getRect
method and the setPosition
and setSize
methods will be replaced by the setRect
method. fullscreen
and minimize
methods will be available, so these actions can be performed within your test.
Other Notable Changes:
Options
class for every browser will now extend the Capabilities
class.driver.switchTo().parentFrame()
method has been added to make frame navigation easier.nice
locators will be included that operate on a higher level to the current ones. They will be a subclass of By
.DevTools
API, allowing users to take advantage of features offered by using the Chrome Debugging Protocol (and equivalents on other browsers). These include:
Note: You can get an Alpha version of Selenium 4 from the Maven repository. It’s highly recommended to try this out against your current framework (ideally on a sandbox branch), so you’re ready for the change.
In this article, I’ve covered a few areas where I’ve improved my test automation framework for the better. It’s now more stable and usable than before, which will have a positive impact on everyone involved in the software delivery lifecycle.
Making the changes that I’ve outlined above is a good start, however, I highly recommend that you review your entire framework for “pain points,” as there may be improvements you can make that I haven’t covered. For example, are you using WebDriver Manager to manage your drivers or are you still updating them manually?
I’d also recommend setting a date to do this at least once a year, although ideally it would be every six months. In the article, I’ve included the changes that are coming in Selenium 4.0. Please do review the alpha
versions of Selenium for yourself. The changes will be dramatic and you’ll need to be prepared. I hope you’ve found this useful. If you discover any new methods or techniques while doing a bit of spring cleaning on your framework, please do share them with other readers of this blog by adding them to the comments section below.
Also, if you want a look at automated tests in Selenium, and how you can use Page Object models to write maintainable and reusable test routines, check out Automation in Selenium: Page Object Model and Page Factory.
Sleep will always pause for a set amount of time before executing some code, while wait will only pause execution until an expected condition occurs or it times out, whichever comes first. You should never use sleep in a test automation framework as you want your tests to run as fast as possible.
This is when you create automated tests using the Java programming language.
In order to write automated test cases for web apps which run functional tests in the browser, you have to use a tool such as the Selenium library, as we have in this article. You will also need to use a test framework such as TestNG or JUnit.
There are many test and assertion frameworks that you can use to run your Selenium-powered automation tests. I use TestNG as it’s specifically designed for Acceptance Tests, while frameworks such as JUnit are commonly used for unit testing. Another great framework that is well worth investigating is Spock as it’s highly expressive and easy to read. Google’s Truth assertion library is also a great way to write readable tests.
Cardiff, United Kingdom
Member since May 21, 2019
Simon has been a technical tester for over 18 years. He started at IBM, moved to EADS, then Fujitsu, and now runs his own company.
PREVIOUSLY AT
World-class articles, delivered weekly.
World-class articles, delivered weekly.
Join the Toptal® community.