The Truth about Testing?

It takes a lot for me to get riled up, but here I am.

Stuart Reid is doing a keynote at EuroSTAR titled “When Passion Obscures The Facts: The Case for Evidence-Based Testing.”

Here are three things he intends to show:

• How testing ‘evangelists’ use their apparent passion to conceal a lack of evidence supporting their claims
• Which claims are supported by evidence, which are just plain wrong, and which lack real evidence.
• How we should collect metrics to provide evidence to support testing improvements.

To me, these are not articles of scientific inquiry for an honest presentation about the origins and intricacies of controversies in our craft, they are weak opening arguments in a frivolous lawsuit he is bringing against it. 

His argument is that there are rival philosophies of testing (called “schools”) that are misleading you about testing. (Though for what purpose, he does not say).  This talk seems to be about how he will drag these rival, passionate evangelist ne’er-do-wells before the High Council so that he can show how they are obscuring the truth as represented by what he calls “facts” & “evidence”.
 
First, I identify myself as one of the “passionate evangelists” from one of the schools he is taking to task (the Context-Driven School). Second, I consider myself an advocate for the craft and science called “software testing” and that questions like “is exploratory testing more effective than scripted testing?” need to take a lot of context into account before they can be answered to someone’s satisfaction.  But to say I have the “facts” about controversial testing topics like this framed as “evidence” that can transcend years of controversy would not only be ridiculous, but arrogant and insulting.

But he goes on…

“This presentation will identify which claims are supported by valid evidence, which claims disagree with the available evidence, and those claims where there is currently insufficient evidence to reasonably support a claim one way or the other.”

Did you notice what words he chose to accompany the word “evidence?” — “real”, “valid”, “available”, and “insufficient”. 

According to whom?  You, dear reader? 

Of course not.  You can’t use these words because you don’t know any better.  You’ve been manipulated.  He hasn’t, thank goodness.

His case depends on convincing you how his evidence — obscured to you by people like me [see his title] — finally allows you to sort out six specific software testing controversies that have persisted for years.  How else other than showing you his briefcase full of facts will me and the other svengali evangelists from rival testing schools be exposed for misleading you about these issues? How else, other than seeing his evidence, will you be free once and for all from the polarizing debates we svengalis perpetuate?

I see Reid as a misguided politician-lawyer who needs a big case to get noticed.  He’s hoping you will not be smart enough to see that any premises (and promises) of “evidence” are subjective.  In other words, they need context — the theme of one of the very schools he says is swaying you. 

Is he really the crime-fighting hero, armed with a briefcase that once opened, would settle these testing debates bewteen the rival schools that have been misleading and plaguing gentle, innocent, unsuspecting tester-folk for years?

I think it’s more likely that you’re the jury in this case, knowing that software testing is a challenging intellectual process, not a set of absolute truths held in someone’s briefcase waiting to be laid out for you — especially by someone who doesn’t think it is. 

At least, that’s what the “evidence” of his title and abstract show to me.  The main difference between me and Reid is that my School has taught me that evidence, like in court of law, can be circumstantial.

Advertisement