I’ve attended a lot of testing talks over the years and there are three things I’ve heard that really bug me. If any of these statements are in your slides for your next talk, let me try to compel you to get rid of them because they may distract testers like me enough to cause us to tune out:
1) “Insanity is doing the same thing over and over and expecting a different result.”
Einstein allegedly said this and it’s used a lot by speakers who want to decry what appear to be silly pathologies of project management.
But wait a second. Let me summon you testers out there to help me examine this.
Have you ever done the same thing over and over, expected a different result and felt *more* sane because of it?
Have you ever done a series of key presses (like a “slot machine” test) and expected to uncover a possible buffer overflow any second? Press Enter <no change>, Press Enter <no change>, Press Enter <no change>, Press Enter <BANG!>. You’re expecting that BANG after many trials of doing the same thing.
This is insanity? No more insane than people who exercise the same way every day and expect to get thinner than they were the day before. No more insane than a fisherman’s wife looking hopefully out to the ocean every day to see the mast of her husband’s ship on the horizon.
Lesson: Things change despite our routines. In fact, it is our routines that catch new behavior from a system that changes underneath and around us. Like a Build Verification Test is meant to be the same thing over and over, hoping to catch the new code doing something it shouldn’t today, routine can help us find bugs.
Let me flip it around… it is a SANE thing to do something over and over and expect a different result.
2) “The longer you wait to fix a bug, the more costly it is.”
I’ve had three experiences in my career where the opposite has been true. Here’s one of them:
I worked on a project whose developer had a bug assigned and open to him for 183 days without any status other than “Yeah, I know about it, but will not be an easy fix …” The pressure mounted week after week, and a few days after his latest one-line status report, the bug was fixed.
There was celebration in the triage committee. After 190 days, it was closed at last! But people were dying to know, how did he do it? Well it just so happened that the best outlook of a fix came in an recently released SDK. He used an API that fixed the problem and the implementation took 30 minutes. If he had jumped right on the bug the day it was opened and spent weeks fixing it, all of that time would have gone to waste.
Lesson: Sometimes procrastination pays off. Projects get cancelled, schedules get flipped, priorities get rearranged — all of this can result in re-work and wasted time. Not always, but neither is it always the case where the sooner you find and fix a bug, the cheaper it is to fix.
3) “When you “assume”, you make an “ass” out of “u” and “me”.
Do you know how many assumptions we make in one hour? It has to be thousands! I assume the sun will go down today, I assume I will still have a job in the next 60 seconds, I assume there will be livable oxygen to breathe by the time this sentence is written…
I do not question these and I am not an ass because of it — and neither are “u”.
If we questioned every assumption, we’d go insane — or at least sound insane.
“I assume that you are reading this sentence. I assume you know that I am a human and that you do not think this sentence was generated by an electronic algorithm. Assuming that, let me now assume that you will allow me to make this next remark in these words that you are choosing to read of your own free will and volition and let me assume that I may have to define “volition” for some of you which I will do now, assuming that you care about such things…”
Lesson: I think the spirit behind the “ass out of u and me” remark can be restated like this:
“If we don’t examine the consequences of some assumptions, we run the risk of looking foolish and losing credibility in our community.”
But which assumptions should be questioned?
I don’t know, but I do have a triggering heuristic that helps me. Whenever I’ve been stuck or frustrated or bored or confused or mad or off-center in some way, I’ve learned that those are good times to question assumptions.
I’ll try to list a few assumptions I have as I test and question one them before my next test and see what happens.
I ran into a bug in a tool my brother and I wrote — a crazy bug since the tool had worked for years the same way and one day it stopped working. I spent 3 hours investigating the problem. No luck. Somehow the tool just got corrupted. I took a break and came back. A heavy sigh later, I started at square one, questioning every step. And there it was.
I assumed I had Perl installed.
No, I could have SWORN I had Perl installed. I remembered installing it the night before. But it turns out I had just downloaded it and never ran the EXE. I never thought to question my memory. Maybe that’s why a few times over the years when testing, I see the error dialog “This bug should never happen.” — a note put in by programmers who perhaps have made those kinds of simple assumptions and have felt the consequences of not questioning some of them.
So, if you’re going to use aphorisms like these in a testing talk, test them first. I’m not annoyed at people who use these in their slides as much as I am annoyed at the people who nod and even laugh when the speaker says them.
Ugh. C’mon, everyone, we’re testers! I’m not suggesting we test *every little thing* that’s said at a talk, but whenever the speaker is trying to provoke or persuade you, it should be a trigger for you to learn why it is you are provoked or persuaded and to test the premise of the remark.
Maybe all it takes to help with this is awareness — which they say is “half the battle.”
Are you thinking “Well, Jon, what’s the *other* half of the battle?” and “Who is the *they* you’re referring to?” and “What does ‘half of a battle’ look like anyway?”
Cool. Then I don’t have to say another word…