Lessons Learned at CAST 2013

Jon Bach Keynote

Testers should jump to conjectures, not to conclusions. – Jon Bach, Keynote, CAST 2013

This week I was fortunate to be a last-minute addition to the attendee list for CAST 2013 in lovely Madison, Wisconsin. Somebody dropped out 3 days before the conference… I found out I was going on Saturday.  I had already planned to run in the Detroit Merrell Down and Dirty Mud Run on Sunday (which is one of these Warrior Dash-style obstacle 5ks where you crawl around on wet road gravel), so I still did that thing.   Then, exhausted already, the skin on my knees completely trashed from the obstacle course, I hopped in the car and drove the 6.5 hours to Madison.


I pulled in to Madison at 10pm Madison time.  I checked in to the hotel, went up and put my stuff away, and checked CAST’s official chat room (Twitter) to see what was up with the testers. They were at the bar. Alright, I’m exhausted to the bone marrow at this point, forget it, I’m going to bed.

“Wait a second. That bar is in this hotel.”

I staggered down to the hotel bar and walked directly into my own Twitter feed.

Long story short: I met a bunch of amazing people at the hotel bar about 5 minutes after I got into Madison, and the trend continued every 5 minutes thereafter (excluding the 5 hours I slept every night) until I finally drove out of town, 4 days later.


I’d never been to a testing conference before, but my overall impression is that it’s a lot of wandering around booths and attending lectures. CAST is not about lectures: it’s about conversations, and some of the best conversations I had were at Lean Coffee, a daily micro-unconference hosted by Pete Walen and Matt Heusser. The basic idea is that you get together with some peers and talk about problems and ideas. It’s a good format; I got a lot of direct, pragmatic advice from some really smart people. Lean Coffee is something pretty cool; I really appreciate being exposed to the concept and plan to explore it further now that I’m back home.


The conference took place at Monona Terrace, which I can only describe as “a conference center designed by Frank Lloyd Wright, where the light in the lunchroom is sunshine reflected from the waves of a beautiful blue lake.” It’s a gorgeous venue. If you’re ever in Madison, or even driving through around mealtime: they have an outdoor cafe-style restaurant on the roof with beautiful views of the city and the lake. Just saying.


After attending Lean Coffee for the first time, I went into the conference with two tasks: choose a tutorial session, and figure whether I was actually allowed to attend sessions in the first place. As I said, I was a last minute replacement, they still had the other guy in the system. Right off the bat, I spoke with conference organizers Ben Yaroch, Dawn Hayes, and Scott Barber about the situation, and they said it was fine. I never did figure out whether I officially “attended” CAST 2013 but they didn’t kick me out.

I ended up attending Robert Sabourin‘s Visual Models In Test Design tutorial, which I selected because it seemed most relevant to me, already having taken Paul Holland‘s course in Rapid Software Testing and not being a mobile tester (which were the other available tutorials).

My mindmap from the first day:
Visual models in test design
It was pretty involved. The tutorial contained a lot of nuts-and-bolts test techniques (detailed in the mindmap), and the techniques were illustrated by testing software for a Lego Mindstorm robot. It was pretty surprising how fun it was to write and run test cases for a toy, and that activity brought me to my first important Lesson Learned:

Lesson: Software testing is about learning, not about proving correctness.

We broke into little groups and created mind maps of potential test ideas for the robot, which basically drives forward until it hits a line on the ground and then follows the line. My group came up with a huge map of potential test ideas, and then asked if we could just go try some to see how the robot behaved, not knowing much about it.

Our first test run proved that all of our test cases were way off, because the robot behaved in ways that we didn’t expect. It turned out that the interesting test space for the robot had to do with behaviors we hadn’t even considered until we ran a test. Our first set of test cases all set to prove that a theoretical robot worked like we hoped it would; that was a waste of our time. We threw out all those tests, started with new ones that explored the robot’s *actual* behavior, and got a lot more productive.

Day 1 ended at State Street Brats with some of our newfound testing friends from the class. Summary: Yummy brats, cheap delicious beer, and no air conditioning. I had some good conversations there, played a little Set, and went home for the night.


Day 2 kicked off with a keynote from none other than Jon Bach, Context-Driven testing guru and author of the original whitepaper describing Session Based Test Management, which is assigned reading for every new employee that I onboard in my organization.

Jon’s keynote address was about the value of argument. Specifically, he dug into the proposition that, in order to move forward, people need to be articulate and forceful about their arguments, without conceding that the other side of the argument is equally valid. We can’t get to better, clearer ideas unless we force ourselves and others to prove that both sides of an argument are *not* equally valid. Jon related this discussion back to his brother James‘s ongoing antagonistic relationship with some of the rest of testing community, and entreated those of us listening to stake our own position and defend it.

Next, I attended a talk given by Jim Holmes and Matt Barcomb entitled “What is software engineering and craftsmanship, and why should I, as a tester, care?” This was all about clean code and why it matters in the world of software development. The purpose of the talk was to discuss patterns and smells, while encouraging testers to participate in code reviews.

LESSON:Armed with some high-level knowledge of design patterns and code smells, testers can be very productive pair partners and code reviewers.

That afternoon, I attended a discussion by Cleveland-based test manager Erik Davis entitled “How to find good testers in the rust belt.” Since I am involved in my own company’s recruitment effort, this was right up my street. The presentation was a firehose of information: Erik presented useful, practical advice at a pace where I could barely keep up. He exploited the Prezi format to create a more dynamic, interesting variant on the classic Powerpoint style presentation. Erik was on Twitter last night asking for feedback, so my feedback is this: the presentation went by in a flash, it was funny and engaging, and it showed me how I can change my behavior to solve a problem, so, that’s pretty good for a lecture about hiring practices.

LESSON: If it’s difficult to find experienced testers, don’t bother looking, just hire smart people and train them.

LESSON: It’s not a bad thing when testers transfer out of testing, because it creates allies, with testing knowledge, in other departments

Next up was a presentation by Michael Larsen (aka TestHead) about the SummerQAmp initiative, a partnership between testing organizations and the federal government that aims to teach high school kids about testing.

The track ends in an internship; the interns’ managers have given largely positive feedback, though they wish that the testers (who start from zero and are given a 4 week crash course) would have more testing experience (ok) and especially test automation experience (lol) when they start.

I was interested to hear both what they choose to teach as an intro to testing – an intro to testing techniques, and intro to “context,” and an intro to basic web technologies (e.g. HTTP, HTML) – and the challenges they were facing. The challenges were mainly pedagogical: they were struggling with some core educational challenges such as tailoring the material to students of different backgrounds, how to run assessments, self-study vs lecture, etc.

LESSON: There are kids out there who want to explore testing as a career and they need help. If you’ve made it this far into an article about CAST 2013, they need your help.

Next up was “Tailoring Your Testing Timespan,” presented by Geordie Keitt. In this talk, Geordie delved into the work of organizational psychologist Dr. Elliott Jaques. Specifically, he discussed the ideas put forth in the book Human Capability: A Study of Individual Potential and Its Application, which proposes that as individuals grow and learn, they become capable of working without guidance for longer periods of time, and that more capable people – to draw a chess analogy – are better able to see all of a larger board than those less capable.

This was, to me, a very provocative session. Mainly, I found problematic the conflation of the concepts of “capability” and “organizational responsibility”: I certainly can accept Jaques’s conclusion that experienced practitioners of a discipline hold more plates in the air and engage in higher-level decisionmaking than novices or thoughtless practitioners… but the idea that people put into strategic roles are necessarily more thoughtful or capable than people in non-strategic roles doesn’t jibe with my experience, and I suspect others would agree.

LESSON: Professional happiness and personal fulfillment comes from having responsibilities that are well-matched to your capabilities.

LESSON: “Mid-life crisis” (per Jaques) occurs when you achieve a higher level of capability and suddenly see in retrospect decisions you would have made differently if you had taken a more strategic view.

Then there were the lightning talks. I tried to sign up for one, but it turned out that I got there too late. The talks themselves were quite interesting, including:

  • Julie Hurst, speaking on the impact lightning talks have had on her professional trajectory
  • Peter (Simon?) Schriver on letting schedules slip to save testers’ sanity
  • a talk about group testing presented by Rachel Carson
  • a talk from Olivier Mireault regarding the value of testers participating in Toastmasters
  • Natalie Bennett on how she sees her testing life as being the beloved weird aunt that exposes her product to unsafe toys
  • a talk from Alessandra “Ale” Moreira about introducing exploratory testing to a Factory testing organization
  • Lanessa Hunter on how she draws inspiration from Gumby in her work life
  • A talk from Jesse Alford that metaphorically connected the concepts of focus in testing and photography
  • A talk from Jon Bach about “Open Book Testing” which is a lightweight exploratory assignment technique meant for group testing
  • Michael Larsen discussing the reasons behind his deep dive into the 99 things You Can Do To Become a Better Tester

They were good talks, I was kicking myself for sleeping on the signup.

LESSON:Being courageous after the deadline doesn’t count.

The day ended with a session presented by Robert Sabourin on Experience Reports. The talk was really just RobSab giving a couple of experience reports about things he did, to show us how he does them. What mainly came through, for me, was “Wow, RobSab is a great speaker.” There’s a lot of… stagecraft there.

Then it was off to Great Dane Brewing Company for the official meetup and then off to bed.


Day 3 kicked off with a keynote from Dawn Hayes about “Introspective Retrospectives,” all about personal growth, recognizing your weaknesses, and working on them. It featured a video which I can only describe as “the most suspenseful movie I’ve seen since The Conjuring.” (You had to be there.)

LESSON: If you don’t know how to do something, try teaching it… and then you might find that you do know how to do it.

Next up was “An Ongoing Journey of Testing Mentorship” with Rob Bowyer and Sabina Simons. In the discussion, they discussed how Sabina grew up through dev, to automation, to finally being a tester, and how picking good mentors (and dismissing bad ones) had helped her grow in her career.

The interesting thing for me, in this session, was that it seemed like the mentor relationship between them had really come to fruition once Sabina became a mentor herself.

LESSON: Mentors need mentors.

My last scheduled session was with Anna Royzman, entitled “Quality Leader: The Changing Role of a Software Tester.” The first half focused on Anna’s transition from a test manager to a “test leader”: that is, someone within the organization that helps people test well. The second half focused on James Bach’s Seven Types of Testers, seeking to put each of us into one of those taxonomies. I found this to be pretty eye-opening: my assumption was – based on my own experiences – that the typical tester was the “Empathetic” type, but in reality there was a pretty even distribution.

The overarching point: if we self-identify with one of the types, we should seek out testers of the opposing type to complement our skills, e.g. empathetic testers should seek out technical testers and vice-versa.

LESSON: “Whatever skills you have, whatever skills you need, you really need to talk to more people about testing.- Anna Royzman, CAST 2013

The final event was a wrapup with Scott Barber and Robert Sabourin, where they shared their favorite lessons from the week (much as I have done here).

BONUS DAY: Test Leadership Camp 2013

In a discussion with Anna Royzman, I had heard that there was an extra session going on the day after the official conference… Test Leadership Camp was a mini-unconference that lasted one day. I was happy to participate in a few sessions, where we discussed, among other things, what being Context-Driven means outside of it being the opposite of Factory testing, ATDD, test management, and the basic question of test management versus test leadership. It was very much worth the extra time and I was glad I stuck around, especially for my last lesson, which Jean Ann Harrison taught me two minutes before I left CAST 2013:

LESSON: If you want the people around you to be more curious, be more curious about the people around you.

-Clint Hoagland (vsComputer)


3 thoughts on “Lessons Learned at CAST 2013

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s