The SpecFlow Chronicles – Volume 4: A Useful Pattern For Maintainable Test Automation

Note: in the context of this article, “test automation” refers to the classic definition of test automation i.e. automatic fact checking.  I sympathize with those who object to the term and the paradigm that follows it.  I do believe that automatic checking has a lot of value, though, and I suspect this article will help more people if they can find it searching for the term “test automation.”

Over the past several years, I and some others have sweated over getting a complex, multiplatform software application wrapped in UI tests. Like a lot of software projects, our test automation started out with a less-than-optimal design, and we’ve improved it quite a bit over the years. Now, when I start new test automation projects, I take the lessons we learned and apply them, and things go a lot better.

The primary complaint about test automation (and UI test automation in particular) is that it hurts team velocity in the long run because it’s hard to maintain. Somebody changes one thing in the UI, 500 tests break, and fixing the tests takes 500 times as long as the change took. This is what we call brittleness: an automation suite is brittle if it cannot adjust quickly to a change in the system under test.

What we’ve learned: It’s possible to minimize brittleness by using the right kinds of abstractions in your test code. So, without further ado, here is a useful pattern for maintainable test automation:
maintainableTests

Let’s start from the bottom and work our way up to the top. The examples are in C# but the pattern is language-agnostic.

UI Maps

This is where you define how your UI test automation framework is going to identify the things that it is going to control. Let’s say that your application is a web app, and the first page you see is called Login.html. The UI map is a class called Login.uimap.cs. The UI map class contains *only* the UI definitions. The tests themselves should not use this class in any way. It should only be used by a Page Object.

Page Objects

Page Objects are classes that expose the controls to other classes. In the Login example, the Page Object would work like so:
public class LoginObject
{
public void LogIn(string username, string password)
{
var loginMap = new LoginMap();
loginMap.Username.Text = username;
loginMap.Password.Text = password;
Mouse.Click(loginMap.LogInButton);
}
}

A n00b mistake: letting your tests interact with the UI map controls directly. That’s how you make brittle tests. But we’re not done, there’s more we can do to make these more maintainable.

Helper Classes

In general, when you’re viewing the application from a user’s perspective, you’re not going to want to think in terms of “pages” and “interactions with pages,” but that’s what you’d have to do if Page Objects were your highest level o abstraction. Instead, I recommend a higher level of abstraction called “helper classes.” These describe workflows through the product:
public class LoginHelper
{
public void LoginToProduct(User user)
{
var login = new LoginObject();
login.NavigateTo();
login.LogIn(user.Username, user.Password);
}
}

The User class in that sample is a Default Data Object.

Default Data Objects

One helpful thing to keep in mind when writing tests is that tests document the product. They can be useful as check against regression, and they can be used to demonstrate that some criteria have been satisfied, but they are extra-useful at documenting the expected behavior of the product.

However, that documenting quality is lost if the tests don’t help a reader understand what the tests are testing. To that end, what you *don’t* want is to have your tests including lots and lots of data that isn’t interesting with regard to the functionality being tested. If the username and password isn’t germane to the example being tested, and you just need to log in for the test to work, then you definitely do not want the test to include a username and a password.

To that end, we create Default Data Objects:
public class User
{
public string Username {get; set;}
public string Password {get; set;}
public User()
{
Username = "vsComputer";
Password = "12345";
}
}

This enables you to just say that var user = new User() and then your test has a user. You only specify the username and password if the test is about usernames and passwords. Default Data Objects radically improve the readability of your tests.

Domain Specific Language

Now, this will be different depending on whether you want to use BDD-style frameworks in your tests or if you just want to make tests. If you *don’t* want to use BDD-style language in your tests, this layer would be your tests. The tests themselves should be a bunch of helper classes consuming and passing around default data objects.

If you’re using BDD frameworks (like Cucumber or SpecFlow), then this layer consists of step definitions. The step definitions should be nice and tight, if you’ve done it right then the step definitions should only contain helper classes working with data objects.

Living Documentation

If you’re doing BDD-style tests, then this layer consists of automated examples.

Using this pattern keeps everything readable, from the automated examples all the way down to the UI maps, and it keeps everything maintainable: if a UI element changes, it’s very easy to fix all of your tests by changing a single UI map. If the whole login page goes away and you now log in some other way, then it’s *still* easy to fix your tests: you just change the workflow in the LoginHelper to reflect the new way it works and throw the Login object and UI map away completely.

I’ve found that using this pattern is really helpful and flexible, but it seems like a LOT of work to put it together. It’s not, though. In my next post, I will show how to use Agile principles and patterns to build this structure up with a minimum of effort.

Advertisements

2 thoughts on “The SpecFlow Chronicles – Volume 4: A Useful Pattern For Maintainable Test Automation

  1. What would Quint or Glenn’s take on this be? I miss post-it’s with your various incarnations. 🙂
    Been trying to track you down Clint. Drop me a line if you have a minute. BTW really like the new Bachelor Machines stuff, but I miss the Fullerene’s. 😉
    Mark Mauren (Former A-O, A.S.S.co-worker/survivor)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s