The SpecFlow Chronicles – Volume 1 : Turning Tests Into Living Documentation

As I mentioned in last week’s post, part of my focus is to help my team increase its capabilities in the area of test automation.  We would like to increase the scope of our automation and, in particular, we would like to increase the level of collaboration between developers and black-box QA in implementing and maintaining the tests.

Some of the challenges that we’re currently facing:

  • Many black box testers lack the automation experience needed to automate the tests that they want to automate
  • Developers don’t know what the testers want to automate
  • The tests that are currently being run are not transparent to the business side
  • Existing test automation is not being maintained or exploited because it’s only well-understood by those that implemented it

Over the past few months, my team has been playing with an open-source tool called SpecFlow as we try to address these challenges.

What is SpecFlow?

SpecFlow is a Visual Studio plugin that lets you bind business-readable acceptance criteria to .NET code. As a tool, it sits in the category called BDD frameworks; – it’s based on a Ruby BDD framwork called Cucumber. If you’re familiar with Cucumber, SpecFlow is Cucumber for .NET developers. If you’re not familiar with Cucumber, read on…

How it works

After you install SpecFlow (which requires a Visual Studio plugin and a NuGet package for your test project), you will see a few new items in your available New Items in Visual Studio:

  • Feature files are a place to specify tests in terms of acceptance criteria.
  • Step Definitions are files that are bound automagically to the text of the acceptance criteria.
  • Hook files are a repository for special setup and teardown code; they are not required.

When you want to make a new test, you write out the test from a user’s perspective in a specific business requirement format called Gherkin. I did not make that name up, do not blame me, blame the open source geeks that made Cucumber.

The tests look like so:

Right now, this isn’t automated at all. How does it get automated?
If I right-click on these non-automated steps, it will generate the code that goes behind them:

Then, somebody that understands C# fills in the steps with code that actually carries out the actions described by the steps.
Now, the special SpecFlow magic is that when you build this, SpecFlow will dig into all the feature files and make unit tests out of all the scenarios. This is important to emphasize: the scenario becomes the unit test, not the step definition. When you build, SpecFlow goes behind the scenes and figures out what step definitions match the scenario steps, and then it builds an automated test by chaining the steps together. This means that, now that “Given I have launched my app” has a definition, if I use that step in another scenario, SpecFlow will automagically wire it to whatever code you used the first time.

This confers numerous advantages:

The tests are way easier to read

Let’s say that you are a tester working for an elite supervillain.  You’re working with the fortress team in implementing the villain’s fortress defenses, and to that end, you’ve written some tests:

Now, I’d say those are pretty readable tests, as these things go. As a person that stares at these things all day long, I can tell what they do pretty well. However, would you hand this to a supervillain?  Probably not, unless they were the kind of supervillain that’s really good with computers like that “I’M INVINCIBLE” guy from GoldenEye.

Here’s how it looks using SpecFlow to bind the test to business-readable steps:

Now, having seen the original code and the way that steps are laid out in step definitions, it’s pretty easy to imagine how these steps are wired up. And because the test is the scenario itself, if the steps do what they say, those english words are really true. If they weren’t true (or if they become untrue), the test would fail. The features become what Gojko Adzic describes as “living documentation.” You can easily show this test automation to anyone irrespective of their technical background.

Anyone can participate in the creation of automated tests

The author of the feature file could be anyone. The step definitions will need to be filled in by someone who knows a bit of automation, but at this point it’s a matter of collaboration between a dev and a tester (or business rep or whomever). In practice, I’ve sat with devs and said “here’s the steps we already had, here’s one that I need your help on” and the devs can knock out those step definitions in minutes, and I’m off and running. It seems like a nice way to work. We’ve only just started, though.

It’s allowed us to eliminate piles and piles of test code

Creating the common codebase in terms of step definitions has radically reduced duplication in our test code. Here’s an example of how SpecFlow can help reduce the total number of lines of code in your tests:

I don’t want to get too far into the deeper features of SpecFlow right now (that will come in future posts!) – suffice to say that, with very little effort and very little additional code, we’ve changed one test case into three test cases that will run back to back.

Please share your experiences and questions in the comments!  Upcoming posts include explanations of how to configure SpecFlow for your project, how to use the hook files, and lots of more specific tips and techniques.


8 thoughts on “The SpecFlow Chronicles – Volume 1 : Turning Tests Into Living Documentation

  1. Pingback: The Tale of The Smoke Detector That Didn’t Beep | Tester Vs Computer

  2. Pingback: The SpecFlow Chronicles – Volume 2: In Defense of ATDD | Tester Vs Computer

  3. Pingback: The SpecFlow Chronicles – Volume 3: A Good Thing About Gherkin | Tester Vs Computer

  4. Pingback: The SpecFlow Chronicles – Volume 4: A Useful Pattern For Maintainable Test Automation | Tester Vs Computer

  5. Pingback: WTF is a Unit Test? | Tester Vs Computer

  6. Pingback: Describing, Not Prescribing: A Different Way Of Looking At Test Documentation | Tester Vs Computer

  7. Pingback: Agile and relevant practices | Rock Paper Linux

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s