Tips to make your (web) application testable

This blog post will provide you some tips to make your (web) applications better testable. This is an initial list of tips and can be extended of course. The idea is that you can use this list when you have discussions with your developers about testability.

  1. Unique identifiers
    Especially when doing UI testing, it’s important to have unique identifiers on the page. There are different approaches: assign an `id` to every element or assign a unique identifier to each component and input element.Implementing id’s for every element is costly and unnecessary.Implementing unique identifiers for every component is a better approach because they can search relatively within that component. Like, you have a search component with an input field and button:

    <section id="search">
     <input type="text" name="query">
     <button type="submit" name="search">Buscar</button>
    <section>
    

    Now you can construct the following CSS locators:
    section#search
    input[name='query']
    button[name='search']

    Although it’s not needed, I’m in favor of defining the type of the element.

    When not having these unique identifiers you will probably end up with very hard to maintain locators. A few examples:

    .//*[@id='wx-header-wrap']/div/div/div/div[2]/div[2]/div/section/div/form/input

    or

    .//*[@id='gnav-header-inner']/div/ul/li[2]/a

    The first is too long and therefor unreadable. The latter is much shorter but is tightly coupled to the second list-item (li[2])

  2. Separate environment
    Once, I was on a project where the test-database was shared with multiple test environments. I believe this was done to save some costs, as we depend on a huge Ora cle database. Cost-saving is fine, but it had some drawbacks.

    • Different testers/teams are manipulating the same data;
    • Different versions of software are processing the same data;
    • Different message-consumers are messing with the data.

    This was not very convenient and after some debate we found the budget to duplicate the environment (including database and all services). The result was an isolated environment for running our automated tests.

  3. Mock third-party services
    When testing in general, depending on data which is outside of your control and become a nightmare. Because of the following:

    • Third-party service can decide to switch-off the servers;
    • Third-party service can decide to clean the (test) database, so the data is not present anymore;
    • Third-party dataset is very limited.

    You might decide to stub/mock the third-party services if you recognize one/some of the above. When stubbing/mocking third-party services you have full control over the data and you can even easily simulate error-responses/timeouts/etc.

It would be great to hear your tip, so I can add them to the list. Feel free to leave a reply (tip + argument and in what situation did you benefit from it).

Share This:

A “Gated” environment

A little bit of context; in my current project we work with 7 scrum teams on the same product. In 2 week sprints, we implement new features on a legacy codebase. We have both manual and automation testers.

It’s a kind of challenge to end up with a repository with functionally broken code when 7 teams committing new code. Over the last 1,5 year, we had quite some issues with people committing SQL errors, conflicting bean names, non-compiling code. This all results in a broken test-environment, frustrations, a lot of wasted time. To overcome this issue, we introduced a ‘Gate’. Once a team tries to merge their team-branch with the sprint-branch, the following actions take place:

Merge –> Build (compile/unit test) –> Deploy –> Functional Tests (API’s) –> Store Build Artifacts

Every step explained:
Merge: merge sprint-branch with team-branch and team-branch with sprint-branch
Build: check if we have compiling code and if the merged code passes our first safety-net, the unit tests
Deploy: artifacts will be deployed to a separate environment with its own database, micro services and even stubs
Functional tests: the deployed application will be tested by our second safety-net, which exists out of end-user flows implemented on an API level (to ensure a fast feedback-cycle).

If one step fails the remaining steps won’t be executed.

For example:
– If one of the unit tests fails, the application won’t get deployed, functionally tested and artifacts won’t be stored
– If one of the functional tests fails, the artifacts won’t be stored

It can be visualized using the Jenkins Build Pipeline plugin:

buildpipeline

So, now we introduced a safety-net and there is an incentive to push functionally working code. The moment existing tests fail due to newly introduced application code, you have full responsibility to repair the existing tests.

The Second Gate…
Now we are sure that all the code merged and tested (on API level) properly. However it can still happen that some bug in the user interface was introduced, therefore we have introduced a second Gate which runs front-end tests. In fact, the following actions take place:

1. The accumulated integrations will be versioned as a release candidate.
2. Release candidate will be deployed to a separate environment.
3. Front-end tests will be run against the release candidate.

Above described process takes place overnight, because these are long running tests. But in fact, it can happen anytime…

I can imagine that you are interested and keen in a ‘Gated’ environment like this. I’m willing to help you with the implementation if you contact me.

Share This:

Disposable Selenium Grid

Over time your Selenium Grid might become unstable due to browsers not being closed properly, browser error popups and so on. This blogpost will cover the creation of a disposable Selenium Grid. Since the introduction of Docker the creation of containers (a kind of virtual machine) became a lot easier. With Docker-compose you can even describe your multi-container setup in a recipe-style providing step-by-step procedures.

  1. Install Docker (https://docs.docker.com/compose/install/)
  2. Put the file below somewhere
  3. Run the following command: docker-compose up

Getting the Selenium Grid Address
Use the following command to get retrieve the IP-address of your Docker container: docker-machine ip
In your script you can use http://[IP]:4444/wd/hub in order to connect to your Selenium Grid.

Scaling you Selenium Grid
With the following command you can easily scale up/down your Selenium Grid
docker-compose scale chromenode=15 firefoxnode=15

Filename: docker-compose.yml

seleniumhub:
  image: selenium/hub:latest
  ports:
    - 4444:4444
  environment:
    - GRID_MAX_SESSION=50

firefoxdebug:
  image: selenium/node-firefox-debug:latest
  environment:
    SCREEN_WIDTH: 2880
    SCREEN_HEIGHT: 1800
  ports:
    - 5900
  links:
    - seleniumhub:hub
firefoxnode:
  image: selenium/node-firefox:latest
  environment:
    SCREEN_WIDTH: 2880
    SCREEN_HEIGHT: 1800
  links:
    - seleniumhub:hub
chromenode:
  image: selenium/node-chrome:latest
  environment:
    SCREEN_WIDTH: 2880
    SCREEN_HEIGHT: 1800
  links:
    - seleniumhub:hub

Share This:

Automatically update drivers

Getting tired of downloading the latest driver into your project? You might want to take a look at gradle-download-task. This plugin allows you to download files (over HTTP and HTTPS) to a specific destination directory.

Herewith a simple example:

/**
 * Include the gradle-download-task plugin
 */
plugins {
    id 'de.undercouch.download' version '3.1.0'
}
 
apply plugin: 'java'
apply plugin: 'eclipse'
 
/**
 * Import the Download task. This line is optional.
 * You can of course also always use the full qualified name
 * when you specify a task of type 'Download'.
 */
import de.undercouch.gradle.tasks.download.Download
 
/**
 * The following two tasks download a ZIP file and extract its
 * contents to the build directory
 */
task downloadZipFile(type: Download) {
    src([
        'http://chromedriver.storage.googleapis.com/2.22/chromedriver_mac32.zip',
        'https://github.com/mozilla/geckodriver/releases/download/v0.9.0/geckodriver-v0.9.0-mac.tar.gz'
    ])
    dest buildDir
    acceptAnyCertificate true
    overwrite true
}
 
task downloadAndUnzipFile(dependsOn: downloadZipFile, type: Copy) {
    from zipTree('build/chromedriver_mac32.zip')
    into 'src/main/resources'
     
    from tarTree('build/geckodriver-v0.9.0-mac.tar.gz')
    into 'src/main/resources'
}
 
defaultTasks 'downloadAndUnzipFile'

You only need to update the version numbers (from time to time) and the files will be downloaded and extracted. (eventually in src/main/resources)

Share This:

Firefox Marionette Driver

As announced some time ago, FirefoxDriver will no longer work after Firefox 47 has been released. There are 2 options;

  1. Downgrade Firefox (version < 47)
  2. Implement Marionette Driver. (Marionette WebDriver)

This blogpost will cover the implementation of Marionette Driver. Luckily it’s not too hard to switch to MarionetteDriver. Following these steps:

  1. Download Marionette Driver
  2. Extract the file
  3. Set the webdriver.gecko.driver property to the location path of Marionette driver
  4. Adjust the code, to something like this:
    	@BeforeClass
    	public void startFirefox() {
    		System.setProperty("webdriver.gecko.driver", "path/to/geckodriver");
    		final WebDriver driver = new FirefoxDriver();
    		driver.get("http://www.google.com");
    	}
    

Alternatively, we can implement a method to find the file on path:

	private static String findFileOnPath(final String fileName) {
		return MarionetteDriverTest.class.getClassLoader().getResource(fileName).getPath();
	}

MarionetteDriverTest is the classname.

And use it like this:

	@BeforeClass
	public void startFirefox() {
		System.setProperty("webdriver.gecko.driver", findFileOnPath("geckodriver")); //assuming the file is located in the resources folder
		final WebDriver driver = new FirefoxDriver();
		driver.get("http://www.google.com");
	}

Share This:

Generate Leanpub Book

While writing a book (Leanpub) it might be useful to quickly replace a lot of variables in one go. Therefore I created this tool. GenerateBook-all-1.0.jar

The tool expects to follow directory structure:

|-orig/ (folder containing the original files)
|-manuscript/ (if this folder does not exist it will be created)
|prop.properties (file containing all variables you want to replace)

Example prop.properties file content:
authors=Author1 and Author2
demo.url=http://example.com
software.version=12.3

Usage:
To use this tool: java -jar GenerateBook-all-1.0.jar

Example output:
Copied: orig/chapter1.md to manuscript/chapter1.md

 

I hope this tool is useful for you and additional features can be requested via comments.

Share This:

2015 Retrospective: a year’s reflection

My first year as an independent software testing professional. This year I worked as a Technical Test Specialist/Coach in the energy branch. My main task was/is to setup a testautomation framework and support 10+ scrum teams with creating automated testscripts.

I experienced/applied some new techniques, like: docker, vagrant, gatling, wiremock.

I initiated an interest group for technical testers. We periodically organize sessions to share knowledge about technical testing (tools / methodologies / practices).

I also achieved the following certificates:

  • Professional Scrum Master (PSM 1)
  • Professional Scrum Developer (PSD 1)
  • Professional Scrum Product Owner (PSPO 1)

Next year I will continue my current assignment.

Share This:

Testing AngularJS applications in plain Java

You might think that protractor is the way to go when testing AngularJS applications… The thing with AngularJS is that you can decide to remove all class and id‘s from the rendered html. (which is good for page rendering times, but bad for testability). In the template files there are some identifiers, but those are not visible in the rendered html, all the webelements with a template identifier have a class with the value ng-binding (or something very similar).

We have a very large broad IT-landscape with a big variety of applications. (JSF (JavaServer Faces) websites, AngularJS websites, JMX services, web services (REST and WSDL), mail server integration, PDF reader, FTP, Ora cle DB and perhaps mobile in the future).

In this case I decided to write a Java-based test automation solution. Protractor is built on top of WebDriverJS. So, I thought the same actions can be done in Java. I found https://github.com/paul-hammant/ngWebDriver which suits our needs. Don’t forget to set the setScriptTimeout(), otherwise it might not work.

Share This:

Argh! What is it with the latest software testing hype??

People tend to use BDD-tools for everything those days… Don’t get me wrong behavior driven development is a great way of software design, but it’s not -just- a testing tool! It’s a way of collaboration and it aims to reduce misunderstandings.

Aslak Hellesøy, Founder Cucumber Ltd,wrote a very good blogpost about it, where he pointed out some very good points. (https://cucumber.pro/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool.html)

When using BDD-tools you’re implementing several extra layers in your testautomation code, which you need to maintain. (Both Gherkin features and step definitions) So, if you use it just for specifying testscript then you introduce extra waste.

Like Aslak already wrote, it can be justified if it improves collaboration and reduces misunderstandings, but if the tool is of testing purpose only those benefits will obviously never happen.

I also like his point about many testers fallen into the same trap…

…they use Cucumber uniquely as a testing tool. No collaboration. No outside-in. They write Cucumber scenarios after the software instead of the other way around. Doing BDD well leads to decoupled software, which again makes it easy to reduce the duration of tests. You don’t get any of these benefits when you treat Cucumber as an afterthought. Personally I have seen many companies using BDD as an afterthought. Often they come up with the argument that ‘everybody can write the feature files’, however they don’t look at the long term effect. Because maintenance can be hell.

I hope you all read the blogpost of Aslak Hellesøy and decide to use BDD in a proper (collaborative) way or decide to not use BDD at all.

It still seems to be a very popular tool, just because things get very readable for non-techie people. They are still a lot of drawbacks: duplicated code, scenario’s written in plain text, extra abstraction layers, etc… JGiven tries to solve some of those drawbacks, as they state on their website:

  • Scenarios are written in plain Java with a fluent Java API – Neither text files, nor Groovy is needed
  • Method names are parsed at runtime and define the scenario text – No duplicate text in annotations is needed
  • Scenarios are executed by either JUnit or TestNG – No extra test runner is needed, thus JGiven works with all existing IDEs and build tools for Java out-of-the-box
  • Scenarios are composed of multiple, reusable so-called Stage classes – No more test code duplication
  • Scenarios and steps can be parameterized for writing data-driven tests
  • JGiven generates HTML reports that can be read and understand by domain experts and serve as a living documentation

So, try to use code driven BDD frameworks, if you have to use BDD-tooling, because you can apply all kinds of development practices. (… and it just makes it easier)

Share This:

Software Testing Pitfalls

In this blogpost I like to discuss some of the software testing pitfalls I encountered over the last few years. Some of them are related to software testing in general, others are related to specifically test automation (Because that is my main area of interest).

Certification
This might be a never ending discussion. My personal opinion; certification is definitely not the guarantee for a good tester, but it allows people to talk a common language. Furthermore, it gives you some proven practices and methods. How much you can use of these methods and practices, depends on the context. I often saw people who became a tester with a background in the area of development or operations. It’s funny if you hear them talk about software testing. Meetings with people without a common knowledge are not efficient at all!

Testing Techniques
I mostly work in environments where we have to test a web application. I often hear tester say that they don’t see the value of applying testing techniques, because it’s just a web application. I think this is really odd, because why should you relate the use of testing techniques to the nature of the application under test (test object)? It makes more sense to relate the use of testing techniques to the business value of the application under test. You have to test more if the business value is high and testing techniques help to produce more efficient test cases.

Testware
You will see it more often that there is less time spent on the preparation and specification phase of software testing. It’s mostly coming from teams who are saying that they apply some form of Agile. I think you should always define some sort of testplan and test cases or test charter, regardless of the development methodology you are using. The level of detail depends on the context where you’re working in.

Test Environments
One of the most heard problems in software testing is the lack of test environments or up-to-date test environments. Utilizing modern tools we can create and distribute test environments on the fly. We also often see that data across environments is totally different. Having those two problems it’s hard to automate testing.

Development Testing
Recently I heard something funny. In a company where they apply Agile, they said: unit testing is up to the developers, we shouldn’t even look at it. (To save time…)

Developers are not testers and (probably) never will be. So, we need to help them with writing proper unit tests. Think of positive and negative cases and even testing all possibilities of a decision. (Also known as: decision testing or multiple decision determination testing) I think, it should also be our task to check periodically the quality of the unit test and verify if they still use testing techniques.

These are just a couple of pitfalls in software testing.

Share This: