Romanian Testing Conference 2017

This year I was invited to do a workshop of my own choice. It turned out to be a “Improve your Selenium WebDriver Testing” workshop. So, I went the 10th of May to Romania to join the conference. The first expression of Romanian people is that they are really friendly, open and honest and luckily their English is also really good. Anyway, those were very good ingredients to start an interactive workshop.

My workshop

There were 32 attendees for this workshop. It was a little bit overwhelming (in a positive way), because normally I put a limit of max. 15. Surprising for me was that half them were woman. (That didn’t happen so far at workshops I gave in the past in mainly The Netherlands.) It went pretty good. I started with a theoretical overview of why and what you should automate. After that we went to the specifics of Selenium WebDriver. All participants enjoyed the locator game, which is available here: http://locator-game.selenium-in-action.io/ . After the lunch, we started to iteratively fix tests and implementing the Page Object Model to achieve better maintainable tests.

 

On my second day of the conference I was delighted to join the presentations of the day. So, herewith a brief summary of the presentations I attended.

Passion wanted to leave me, but I convinced it to stay, by Santhosh Tuppad (Twitter: @santhoshst)

A presentation presented with a lot of passion. He started his career with a few one-week jobs, before he found his passion as a software tester. He said: ‘you need to find your passion, because the passion doesn’t find you’. I think he is very right with this statement.

Testing the energetic consumption of software: why and how, by Paulo Matos (Twitter: )

Poor developed applications consume more energy and they can drain your batteries. (from your phone/ iPad / smart watch). Energy might be cheap in the Western world, but it isn’t in for example Africa or remote areas. Also the batteries of mobile devices are very limited. So, it might be good to optimize your application in regards of the energy consumption, especially when developing mobile applications. CPU/GPU cycles are the most expensive. After all he demonstrated some tooling you could use to visualize the energy consumption per application.

Succeeding as an introvert, by Elizabeth Zagroba (Twitter: @ezagroba ‏)

A nice presentation about how to survive in a project/organization as an introvert person. Surrounding people should be aware how to react to introvert people. (they want keep asking things to identify all possible risks up front) Funny thing was that I actually recognized a lot of the things she said.

Debugging your test team, by Keith Klain (Twitter: @KeithKlain )

One of the points he made was to fire all test managers who spend more than 25% on non-testing activities. Like the typical spreadsheet managers 🙂

 

Test automation – the bitter truth, by Viktor Slavchev (Twitter: @Mr_Slavchev)

He nailed the points on why automating everything is impossible or inefficient, really spot on. He also pointed out that we need to change the definitions, in order to set the correct expectations. “Programmatic testing” in stead of “test automation”. (so, managers don’t expect a huge cost saving and traditional testers don’t expect that their testing will be automated.

Independent tester – A game changer, by Uros Stanisic (Twitter: )

Organizations are requesting more and more T-shaped testers. He described what value a tester can add other than ‘just’ testing.

12 vs. 18, who says “we” cannot be “them”? , by Harry Girlea

This was an amazing talk from a 12-year-old boy standing in front of about 500 people. He pointed out that children start early with playing games, nevertheless they are never part of testing (because they are legally too young). A solution he presented that they (kids) can be paid with game points.

 

All in all an awesome conference! Very eager to attend next year as well.

 

Share This:

Tips to make your (web) application testable

This blogpost will provide you some tips to make your (web) applications better testable. This is an initial list of tips and can be extended of course. The idea is that you can use this list when you have discussions with your developers about testability.

  1. Unique identifiers
    Especially when doing UI testing, its important to have unique identifiers on the page. There are different approaches: assign an `id` to every element or assign an unique identifier to each component and input element.

    Implementing id’s for every element is costly and unnecessary.

    Implementing unique identifiers for every component is a better approach, because the can search relatively within that component. Like, you have a search component with a input field and button:

    <section id="search">
     <input type="text" name="query">
     <button type="submit" name="search">Buscar</button>
    <section>
    

    Now you can construct the following CSS locators:
    section#search
    input[name='query']
    button[name='search']

    Although it’s not needed, I’m in favor of defining the type of the element.

    When not having these unique identifiers you will probably end up with very hard to maintain locators. A few examples:

    .//*[@id='wx-header-wrap']/div/div/div/div[2]/div[2]/div/section/div/form/input

    or

    .//*[@id='gnav-header-inner']/div/ul/li[2]/a

    The first is too long and therefor unreadable. The latter is much shorter but is tightly coupled to the second list-item (li[2])

  2. Separate environment
    Once, I was in a project where the test-database was shared with multiple test environments. I believe this was done to save some costs, as we depend on a very huge O racle database. Cost-saving is fine, but it had some drawbacks.

    • Different testers/teams are manipulation the same data;
    • Different versions of software are processing the same data;
    • Different message-consumers are messing with the data.

    This was not very convenient and after some debate we found the budget to duplicate the environment (including database and all services). The result was an isolated environment for running our automated tests.

  3. Mock third-party services
    When testing in general, depending on data which is outside of your control an become a nightmare. Because of the following:

    • Third-party service can decide to switch-off the servers;
    • Third-party service can decide to clean the (test) database, so the data is not present anymore;
    • Third-party dataset is very limited.

    You might decide to stub/mock the third-party services, if you recognize one/some of the above. When stubbing/mocking third-party services you have full control over the data and you can even easily simulate error-responses/timeouts/etc.

It would be great to hear your tip, so I can add them to the list. Feel free to leave a reply (tip + argument and in what situation did you benefit from it).

Share This:

Automating Jumbo App – Shopping Apple Pie

This Testing Showcase demonstrates the automation of the Jumbo App (supermarket) for Android. The Jumbo App is automated with Appium.

Share This:

A “Gated” environment

A little bit of context; in my current project we work with 7 scrum teams on the same product. In 2 week sprints we implement new features on a legacy codebase. We have both manual and automation testers.

It’s a kind of challenge to end up with repository with functionally broken code when 7 teams committing new code. Over the last 1,5 year we had quite some issues with people committing SQL errors, conflicting bean names, non-compiling code. This all results in a broken test-environment, frustrations, a lot of wasted time. To overcome this issue, we introduced a ‘Gate’. Once a team tries to merge their team-branch with the sprint-branch, the following actions take place:

Merge –> Build (compile/unit test) –> Deploy –> Functional Tests (API’s) –> Store Build Artifacts

Every step explained:
Merge: merge sprint-branch with team-branch and team-branch with sprint-branch
Build: check if we have compiling code and if the merged code passes our first safety-net, the unit tests
Deploy: artifacts will be deployed to a separate environment with its own database, microservices and even stubs
Functional tests: deployed application will be tested by our second safety-net, which exists out of end-user flows implemented on an API level (to ensure a fast feedback-cycle).

If one step fails the remaining steps won’t be executed.

For example:
– If one of the unit tests fails, the application won’t get deployed, functionally tested and artifacts won’t be stored
– If one of the functional tests fails, the artifacts won’t be stored

It can be visualized using the Jenkins Build Pipeline plugin:

buildpipeline

So, now we introduced a safety-net and there is an incentive to push functionally working code. The moment existing tests fail due to newly introduced application code, you have full responsibility to repair the existing tests.

The Second Gate…
Now we are sure that all the code merged and tested (on API level) properly. However it can still happen that some bug in the user interface was introduced, therefore we have introduced a second Gate which runs front-end tests. In fact the following actions take place:

1. The accumulated integration’s will be versioned as a release candidate.
2. Release candidate will be deployed to a separate environment.
3. Front-end tests will be run against the release candidate.

Above described process takes place overnight, because these are long running tests. But in fact, it can happen any time…

I can imagine that you are interested and keen in a ‘Gated’ environment like this. I’m willing to help you with the implementation if you contact me.

Share This:

Automating NS Reisplanner App

This Testing Showcase demonstrates the automation of the NS Reisplanner App for Android. The NS Reisplanner App is automated with Appium.

Share This:

Automating ParkMobile App

This Testing Showcase demonstrates the automation of the ParkMobile App for Android.

Share This:

Clean(er) Test Automation Code

This blog post tries to explain why it is important to really care about your test (automation) code and write clean (automation) code. I’m mainly involved in test automation, so some of the examples given are related to that, but most are related to programming in general. Often test automation code is less exposed to a formal review process. Keep in mind; test automation is code and should threated like that.

I hope this blog post will bring you one step closer to clean(er) code.

Test Class Naming
Defining a classname for unit testing is rather easy; it’s the same name as the implementation class. In terms of functional or integration testing, the classname should reflect the feature or functionality you are testing.

For example:

public class LoginTest { }

public class RegistrationTest { }

public class OrderTest { }

Test Method Naming
I have been on many projects, some Greenfield* others continuing development on an existing codebase. Every project has its own naming convention* or not ☺. The following can happen in projects without naming convention:

@Test
public void test1() { }

@Test
public void test2() { }

// or:

@Test
public void testLogin() { }

Do you get the point? It’s all very descriptive, isn’t it?

Imagine you have to share this with a colleague or you have to present the results to non-tech colleagues, they have no clue what’s going on.

I very much like the naming convention introduced by Roy Osherove (http://osherove.com/blog/2005/4/3/naming-standards-for-unit-tests.html). The basic idea to describe the tested method, expected input or state and the expected behaviour as the name of the test method.

public void nonExistingCredentialsGiven_ShouldThrowException() {

}

We can adopt this principle for feature testing as well.

public void nonExistingCredentialsGiven_ShouldShowErrorMessage() {

}

By following this naming convention we can make sure that the intent of the test is clear to everyone. (Off course the implementation should reflect the method name) Method names are a bit longer, but self-explanatory.

Naming of fields and local variables

Another thing that strikes me is the use of non-explicit fields or variable name. Like:

i for loops
page when applying the Page Object Model (everything is a page lol)
And so on..

We need to be more explicit if we want to create readable code.

Magic numbers
Magic numbers indicates the direct usage of a number in your code. By doing this, your code becomes less readable and harder to maintain.

Example implementing magic numbers:

public class Username {

	private String setUsername;

	public void setUsername(final String username) {
		if (username.length() > 10) {
			throw new IllegalArgumentException("username");
		}
		this.setUsername = username;
	}

}

After refactoring

public class Username {

	private static final int MAX_USERNAME_SIZE = 10;
	private String setUsername;

	public void setUsername(final String username) {
		if (username.length() > MAX_USERNAME_SIZE) {
			throw new IllegalArgumentException("username");
		}
		this.setUsername = username;
	}

}

The refactored example allows you to easily update the maximum size of a username, even if its used in different methods.

A general rule of thumb might be when applying any naming convention is to be very explicit.

Greenfield: setup a project from scratch.
Naming convention: set of rules

Other Test Naming Conventions: https://dzone.com/articles/7-popular-unit-test-naming

Share This:

Disposable Selenium Grid

Over time your Selenium Grid might become unstable due to browsers not being closed properly, browser error popups and so on. This blogpost will cover the creation of a disposable Selenium Grid. Since the introduction of Docker the creation of containers (a kind of virtual machine) became a lot easier. With Docker-compose you can even describe your multi-container setup in a recipe-style providing step-by-step procedures.

  1. Install Docker (https://docs.docker.com/compose/install/)
  2. Put the file below somewhere
  3. Run the following command: docker-compose up

Getting the Selenium Grid Address
Use the following command to get retrieve the IP-address of your Docker container: docker-machine ip
In your script you can use http://[IP]:4444/wd/hub in order to connect to your Selenium Grid.

Scaling you Selenium Grid
With the following command you can easily scale up/down your Selenium Grid
docker-compose scale chromenode=15 firefoxnode=15

Filename: docker-compose.yml

seleniumhub:
  image: selenium/hub:latest
  ports:
    - 4444:4444
  environment:
    - GRID_MAX_SESSION=50

firefoxdebug:
  image: selenium/node-firefox-debug:latest
  environment:
    SCREEN_WIDTH: 2880
    SCREEN_HEIGHT: 1800
  ports:
    - 5900
  links:
    - seleniumhub:hub
firefoxnode:
  image: selenium/node-firefox:latest
  environment:
    SCREEN_WIDTH: 2880
    SCREEN_HEIGHT: 1800
  links:
    - seleniumhub:hub
chromenode:
  image: selenium/node-chrome:latest
  environment:
    SCREEN_WIDTH: 2880
    SCREEN_HEIGHT: 1800
  links:
    - seleniumhub:hub

Share This:

Automatically update drivers

Getting tired of downloading the latest driver into your project? You might want to take a look at gradle-download-task. This plugin allows you to download files (over HTTP and HTTPS) to a specific destination directory.

Herewith a simple example:

/**
 * Include the gradle-download-task plugin
 */
plugins {
    id 'de.undercouch.download' version '3.1.0'
}
 
apply plugin: 'java'
apply plugin: 'eclipse'
 
/**
 * Import the Download task. This line is optional.
 * You can of course also always use the full qualified name
 * when you specify a task of type 'Download'.
 */
import de.undercouch.gradle.tasks.download.Download
 
/**
 * The following two tasks download a ZIP file and extract its
 * contents to the build directory
 */
task downloadZipFile(type: Download) {
    src([
        'http://chromedriver.storage.googleapis.com/2.22/chromedriver_mac32.zip',
        'https://github.com/mozilla/geckodriver/releases/download/v0.9.0/geckodriver-v0.9.0-mac.tar.gz'
    ])
    dest buildDir
    acceptAnyCertificate true
    overwrite true
}
 
task downloadAndUnzipFile(dependsOn: downloadZipFile, type: Copy) {
    from zipTree('build/chromedriver_mac32.zip')
    into 'src/main/resources'
     
    from tarTree('build/geckodriver-v0.9.0-mac.tar.gz')
    into 'src/main/resources'
}
 
defaultTasks 'downloadAndUnzipFile'

You only need to update the version numbers (from time to time) and the files will be downloaded and extracted. (eventually in src/main/resources)

Share This:

Firefox Marionette Driver

As announced some time ago, FirefoxDriver will nolonger work after Firefox 47 has been released. There are 2 options;

  1. Downgrade Firefox (version < 47)
  2. Implement Marionette Driver. (Marionette WebDriver)

This blogpost will cover the implementation of Marionette Driver. Luckily it’s not too hard to switch to MarionetteDriver. Following these steps:

  1. Download Marionette Driver
  2. Extract the file
  3. Set the webdriver.gecko.driver property to the location path of Marionette driver
  4. Adjust the code, to something like this:
    	@BeforeClass
    	public void startFirefox() {
    		System.setProperty("webdriver.gecko.driver", "path/to/geckodriver");
    		final WebDriver driver = new FirefoxDriver();
    		driver.get("http://www.google.com");
    	}
    

Alternatively we can implement a method to find the file on path:

	private static String findFileOnPath(final String fileName) {
		return MarionetteDriverTest.class.getClassLoader().getResource(fileName).getPath();
	}

MarionetteDriverTest is the classname.

And use it like this:

	@BeforeClass
	public void startFirefox() {
		System.setProperty("webdriver.gecko.driver", findFileOnPath("geckodriver")); //assuming the file is located in the resources folder
		final WebDriver driver = new FirefoxDriver();
		driver.get("http://www.google.com");
	}

Share This: