My brother, the Tester

Sometimes when James and I do a talk together, he introduces us this way:

“I’m James and this is my brother Jon. You can tell us apart because I’m the one with hair and Jon is the one who has the capacity to love…”

It’s usually a laugh line, and we like that. It sets the tone of him as Scrutinizer and me as Facilitator. It tells the audience we’ve done this before, we are aware of each other’s reputations, and we want you to be aware, too.

James introduces us this way because he knows that my reputation over 21 years in testing is one of being a “nice guy”. He knows it’s helped me thrive in everyday work environments that would drive him insane. For that, I have his respect.

He knows my brand was founded on helping people cope with imperfect testing techniques, rhetoric, and doctrine. His brand was built on exposing the dangers of those and starting crusades that promoted critical thinking and craftsmanship to fight them.

James may come across to “nice” people like me as being mean because we might think it rude for someone to put our beliefs and assumptions in the spotlight, because we do not. They might think conferences are for gentle, quiet learning, like going to a mall on a quiet Monday morning and doing some window shopping.

To others like James, conferences are an arena where ideas (and people) are examined to either forge them in fire or burn them to ash as frauds.

I think it can be both, but notice that the element of heat or fire is key in both analogies. Heat or fire means examination in public view. Some will take the opportunity to let that fire forge them.  For others, the fire burns and leaves scars.

I am a quiet guy, but I do see the danger in the more “quieter” conferences where speakers simply parrot doctrine to those who are desperate for simple solutions. And why not? When a speaker on a stage has an answer to a problem, they feel smart and conferences are a great way to look smart.  When they get scrutinized, they might not look so smart.

But that’s testing.

Testing is not the search for simple solutions (even though it seems that’s where we’re headed these days). Testing is about scrutiny.

That’s who James is. He is a tester.  He tests things by putting them in a spotlight. He tests people, too — their ideas and rationale. For those who take a stage to do this, the heat from this can be especially strong.

I know this because that’s what happens to me around James sometimes.

I FEEL TESTED.

Actually, it’s not that I feel tested, but that I feel STUPID when I can’t pass the test.

That’s a problem *I* need to solve, not in handling being tested, but FEELING a certain way about it. After all, when I pass tests, I feel pretty good and take the credit, so it’s up to me to handle feeling bad.

There are times I felt stupid in public when I didn’t answer a question to James’ satisfaction, and I always respond professionally in the moment, then take it offline with him later. We talk about what happened, but we also talk about how I felt.  I own where that feeling comes from. That’s the key.

James will be who he is and I have to be who I am, so solving the problem means both of us need to understand what’s going on for each other — how I annoyed or vexed him in that moment and how his vexation triggered ME.

I’ve noticed that once I’ve thought about how I feel and take ownership of that, James immediately turns into a coach. I can also get him to soften by asking for his help in processing how I feel about how he treated me. In these times, he is masterful at helping me NOT feel stupid and focusing on what my strength is, even though he was the one who put me in hot water.

You might think that’s just a brother-to-brother thing, but I’ve seen him do this when he coaches others — men and women, newbies and veterans, people from Sweden and India and China and Israel and Brooklyn.  As much as he questions, he catches people doing something right.  And for those that reach a kind of excellence he can respect, they actually become part of his “team” — his army of colleagues to fight malaise and shallowness in our practice.

He is a teacher like Yoda or Professor Kingsfield (opening clip from “The Paper Chase”) — who in their scrutiny might be downright rude because of the way we feel about it.

When James was my boss in 2000, I often would feel bad for not being good enough. There were many dimensions to his scrutiny because it was his consulting business I was a part of, and many details were important to get right. What we sold was our reputation for excellent testing.

But as much pressure as I felt, James would also find ways to make me better (with my permission), sending me to conferences like STAR West to practice being clear and focused and confident in my slides and narrative as I practice presenting ideas. That same year, I went to Weinberg’s 3-day Problem Solving Leadership seminar where I found other people like me and learned about work by Virginia Satir — work James had studied, too, which made him better in empathy and conflict and in knowing himself.

I read his blog 3 times carefully and I think James is demonstrating what Satir would call “congruence”:

The Congruent Survival Stance

“The ultimate goal of the Satir growth model is congruence. Satir held that high self-worth, self-esteem and congruence are the main “indicators of more fully functioning human beings.” The congruent person holds equal balance in terms of self, others, and context. “When we decide to respond congruently, it is not because we want to win, to control another person or a situation, to defend ourselves, or to ignore other people. Choosing congruence means choosing to be ourselves, to relate and contact others, and to connect with people directly.”

Congruence doesn’t mean being nice, it means being true-to-self, seeing others in the arena of conflict, and considering context and behavior.

Long ago, James gave me something useful when I got annoyed at people: “What are 3 ways this person’s irritating behavior may be entirely rational?”

That line of thinking is something that a Donald Trump does NOT do.

For example, I see from James’ recent blog that he did not show any of the 4 dysfunctional archetypes like Blaming, Placating, Irrelevant, or Super-reasonable and was balancing himself, others, and the context of a situation that unfolded with Maaret.

His work lately has been helping testers reinvent themselves in a belief about Testing that is increasingly written off as DevOps or CICD or “Shift Left” — simple requirements-checking sold as “excellence” or “velocity” or “business value”.

This work started when he and I were talking about my roles at eBay and how I’m not in a defined testing role anymore. He helped me see that I naturally moved into areas tangential to testing or with aspects of testing like analysis, reporting, risk awareness, and problem management. He helped me not be ashamed of this, and see that each of my roles had important testing aspects I could coach others to do (or be aware of).

We started that work together, last year, showing others the value of their work in testing despite the shifting landscape in their organizations that treated testing as a checkbox any programmer could tick. Some on Twitter have compared James to Trump, and I can understand that because James is polarizing. He knows that, too.

But I don’t think Trump can see any merit in any other perspective but his, and his followers have strong emotions that tie them to him. (Trump’s opponents have strong emotion, too, which make them repel.)

People who know MY body of work in testing also know I am no stranger to debate. In years past, I have called out people by name, and used their names on slides, with their quotes, without their permission, to show everyone why I was stirred to action.

In a keynote years ago, I surgically deconstructed quotes from James Whitaker, who had written about how testing was dead. I didn’t merely disagree, I wanted to find those in the audience who needed to see what he said and learn how I felt about it. I got cheers from the audience, and it set a strong thread in other talks at that conference… “In Jon’s keynote yesterday, you heard… ”  That’s what a good keynote should do.

I was proud of that and I still am.

I find it ironic that some who are chastising him for using Maaret’s name without her permission (as I just did there) are invoking his name without his permission, in a public space.  Instead of flaming you for that (another fire analogy), I understand it. You have strong emotion about what he did. That you don’t find this ironic as I do is ok. I see how you might be triggered from it because it’s something in your programming. It’s not a bug, it just IS. But that’s not James’ problem. He’s just the Tester.  You’re the Tested. And just like on a software project, what the Stakeholders do with the information testers give them is up to them.

We can complain they’re not as conscious of the risks of ignoring bugs, but that just means we have an opportunity to do a better job on showing them what we’re really afraid of.

I urge you to do that here.

I learned from somewhere long ago that all Anger is Fear, and all Fear is Fear of Losing Something.

If you’re provoked or angry at him for using Maaret’s name on stage, what is it he’s taken from you?  Peace in our profession?  The reputation that testers are always nice people? Hope for mankind?

If you feel like responding, it’s probably because it tested you in some way that revealed something important.  Whether that’s good or bad is something interesting we could talk about. The merit of a discussion like that is why I named this blog what it is.

Advertisement

Working Towards Precision

It sounds crazy (maybe even arrogant), but the greatest epiphany I had at the recent “Let’s Test” conference in Runo, Sweden came from my own keynote — while I was on stage.  

I thought I was taking a “Critical Look at Best Practices”.  Two days earlier, I had hosted a workshop with 16 colleagues brainstorming, refining, and scrutinizing so-called “best practices” to not only see how they could never be “best” without some context, but to see how they could go terribly wrong in context. After all, this was a conference devoted to context-driven thinking. I was about to share the results, and knew it would be fun to see the reaction of the crowd.

The promise I had made was this:

“You may have heard some software development activities referred to as “Best Practices”, as in “it’s a best practice to write detailed specifications before programming starts” or “it’s a best practice to write a failed unit test and make sure it passes before giving to Test”. In this keynote, Jon Bach talks about the assumptions, risks, and considerations your colleagues think you should consider before using or recommending any so called “best” practice. He’ll share results of an all-day workshop he hosted at Let’s Test the day before this talk where he collaborated with attendees to examine several different notions of “best practices.”

So, armed with the details from the workshop, I delivered my keynote.

“Here are 64 different so-called “best practices” we came up with,” I said, showing the list over several slides, making sure not to say anything and to linger on each side so it could be read fully by the crowd.  

 

• Learn how to tell a good testing story
• Use a test management system
• Carefully choose testing tools
• Vary your test techniques
• Bug advocacy: learn how to sell bugs
• Work smarter, not harder
• Make it clear to PM how much it costs to test or deliver proper software
• Have Dev test it first before it goes to Test
• Don’t spend time doing things a machine can do better
• Make sure test coverage is approved by a stakeholder
• Plan the test environment and data needs early
• Document enough detail to save effort
• Work as a team an communicate
• Design code to be maintainable
• Use exploratory testing
• Test early
• Be able to explain the value you add
• Work with Developers to know who tested what
• Work with Dev to build in testability and logging
• Weak or unclear requirements will cause problems
• Prioritize
• Always add time to estimates
• Be aware of your assumptions
• Certify the testers
• Do research — self education is important
• Become part of the community
• Create clear exit / entry / stop criteria
• Give yourself more time
• You can always find bugs in your software
• Be aware (observation vs. reference)
• Treat bugs as something positive
• Be aware that metrics can be dangerous
• Be aware of pitfalls when communicating metrics
• Talk the same language (“test”, “integration”)
• Don’t let Dev verify bugs
• Use Session-Based Test Management
• Test software
• Check fixed bugs against new versions
• Ask questions
• Send testers to Per Scholas
• Never trust developers
• Talk to each other
• Don’t plan too much; execute as well
• Get feedback often
• Write clear descriptions of bugs
• Use daily standups
• Automate
• Use prototypes
• Communicate with end users
• Consult a variety of stakeholders
• Get good at using your product
• Be aware of your emotional responses
• Don’t be afraid of failing
• Gain and apply experience
• Look out for the unexpected
• Assume specs are incomplete
• Expect that you will never have time enough
• Focus and de-focus
• Think critically
• Use checklists
• Perform smoke tests first
• Use a bug-tracking system
• Discuss issues with Developers first before reporting
• Try to be effective and efficient

I was about to show the top ten according to the workshop attendees, then share what happened when I told them to find how each one of those ten could go wrong.

But before I could do that, my brother James spoke from one of the front tables.

“These aren’t practices,” he said.

James is known for argument, but no matter what he had in mind to say next, I knew I could meet the challenge.

All I had to do to win this argument with him was to point to my definition slide: “Practice: the actual application or use of an idea, belief, or method as opposed to theories about such application or use.”  

I could shut him down by expressing that since anything on the list could be used, applied, and practiced, it was… well… a practice!

In fact, I could even poke fun at the word “used”, meaning while any could be useful (i.e. “able to be used”), they might not all be useful (i.e. valuable or effective).  In those few milliseconds, I felt ready for him.

“These are all vague notions of practices,” he said.

And I realized right there on stage, what he meant.  If I followed through on showing the definition slide, I’d actually be making his point for him.

It was all in the definition… the PRECISION of the definition, the SEMANTICS of the definition, my PERCEPTION of the definition, the MEANING the word had for me, the INFERENCES drawn by the definition and ASSUMPTIONS inherent in the definition, and maybe dozens of other lingual and lexical aspects to consider when even attempting to describe any notion of a practice.

Checkmate, James.  

Immediately, I knew I had another keynote.  I could take these same vague notions of “practices” and turn them into a workshop of how many nuances there are when we communicate with each other.  Or more importantly (and correctly), how we only think we’re communicating with each other. We often don’t take the time to drill down on what we really mean. It’s time-consuming and exhausting to undergo such scrutiny, and in my experience, few people outside a testing conference has the patience for it.

I’m reminded of Michael Bolton’s incredible library of blog entries, in which he has written more than once about language.

Here is a recent gem from March: 

“To me, when people say “it works”, they really mean:

Some aspect
of some feature
or some function
appeared 
to meet some requirement
to some degree
based on some theory
and based on some observation
that some agent made
under some conditions
once 
or maybe more.”

Yes, exactly. It is exhausting to talk in this way, but these nuances matter.

It reminded me of a blog post my brother did when talking about the marketing of “two scoops of raisins” for Raisin Bran cereal and what “two scoops” could really mean.

I could summarize this epiphany (and likely will in a future talk), but for now I’ll let Michael do that for me here with this excerpt:

“Just as we need tests that are specific to a given context, we need terms that are that way too. So instead of focusing training on memorizing glossary entries, let’s teach testers more about the relationships between words and ideas. Let’s challenge each other to ask better questions about the language we’re using, and how it might be fooling us.”

Yes.  And if I may say so, “well said”. I know what you mean.

Crash-Tested

My Dad was in a crash yesterday when landing his airplane. 

He survived and is recovering in a Seattle hospital as I write this. 

When my brother Rob told me, I wanted details, and they were sketchy.  He crashed in Lake Whatcom, no, he crashed in a field in Eastsound.  He was alone, no, he was with another pilot.  He was in a coma, no, just sedated. 

I guess like everyone else, I wanted enough information to decide if he was going to be ok.  Maybe the sooner I knew he was going to be ok, the sooner I would know if *I* was going to be ok.

And that’s what many people around the world want right now with their well wishes and prayers for Dad.

Actually, they can find the answer at the heart of every book he has written.

“Am I going to be ok?”  Flip to the end, the answer is usually “yes”, even if the main character dies a few pages before reaching the back cover.

I think Dad would agree that whether the answer to “Am I going to be ok?” is yes or no, lies a test.  It’s not the answer that matters, but the pursuit of the answer.

Testing is that pursuit.

Dad knows I have made a great living as a software tester for a long time, and he thinks it a fitting career choice, especially given my seagull namesake.

“What is the truth?” or “What might the truth be under different circumstances?” is akin to “Where are the bugs?” or “How might this software cause problems for our customers?”

Seventeen years as a software tester has taught me that a problem is the difference between what you want and what you get.  If they are the same, there’s no problem. 

There are so many quotes from Dad that I could use in metaphor for this, but the ones that come to mind are from Illusions, particularly these excerpts from the “Messiah’s Handbook”:

There is no such thing as a problem without a gift for you in its hands.

and

Here is a test to find whether your mission on earth is finished: If you’re alive, it isn’t.

Dad is alive.

I credit him with having taught me the value of seeing Coincidence.

And, true-to-form, as soon as I typed that, I stopped to give my eyes a rest from the screen.

I looked up at the TV. 

CNN was playing an interview with George Bush senior (president from ’88 to ’92).

They showed him driving a boat very fast, and he said:

“I’ve been in boats all my life. You learn the currents, you learn the shoal waters. That’s where I’m at peace, I just love it. With boats, I’m still in the game. I’m privileged to have a very fast boat a very powerful boat — everyone wants to go on it — it’s a wonderful, wonderful outlet for me.”

For my Dad, it is airplanes and writing.

Now that he’s injured, his ability to do those is in jeopardy.

There’s a test for him.

I don’t know if he’ll be ok, or to what extent, but there’s comfort for me in knowing he’s made a good living writing stories that show the world that the answer can be “yes”.

Foreign Exchanges

It started with this tweet from Ajay Balamurugadas (@ajay184f):

“I use Texter to write the steps for my bugs. Jing to capture screenshots. Brain to think new test ideas. Freemind to note the ideas #testing”

I hadn’t heard of Texter or Jing or Brain. And because Ajay has credibility with me, I quickly ALT-Tab’ed out of TweetDeck, clicked on my XP toolbar to launch Chrome (which has the “WebPage Screenshot” plug-in next to the URL-finder) typed in Google, and started searching.

In that two-paragraph story above, did you notice there were 9 tools?  It made me think about writing this blog. 

How many times have you been to a co-worker’s cubicle or seen someone do a really good demo at a conference and stand in awe and excitement at a tiny little thing they did on their keyboard without thinking about it — maybe something that had NOTHING to do with the demo itself?

That happened for me back in 2006, when I saw someone using notepad, which I had used for years.  They hit F5 when they were typing test notes.  (I’m not going to tell you what it does, so you get the same A-Ha moment when you try it.)

We trade ideas all day on Twitter, in meetings, in email, and in blogs, but rarely in any of those media do we actually SEE each other test, or even type.  The means by which we use our computers in testing is usually hidden to each other.

I found Jing and Texter right away and found value in them, making Ajay even MORE credible to me.

That’s what colleagues do — they help each other solve little testing problems fast with tools that are either free or easy.

But I also like the way Ajay fit that into a tweet.  It read to me like a user story in a way.

So I tweeted back:

“I use EE4 for screencasts, typewith.me to collaborate w/ remote testers, Dropbox for file sharing, Rapid Reporter for quick ET demos. #testing”

Ajay soon tweeted another:

“I use MindMeister for mind map collaboration, Skype to get coached, Perlclip to generate text, Color Cop to identify RGB values. #testing”

I suspected he and I could go several rounds like this, leaving each other with more than a handful of good ideas.

But before I could even think to tweet that, he said it better:

“We exchange a dollar, we still have a dollar. We exchange an idea, we have two ideas now 🙂 cc @jbtestpilot Thanks for sharing tool names.”

I encouraged him to write a blog about any of the other tools he used, and I promised I’d do the same:

He did.  

My list of tools these days is mostly eBay-specific and were built-in-house. They include dependency mappers, perf counters that monitor key site functions (Bids, Checkouts, Listings, etc), a searchable index of customer comments, alerts in our Knowledge Base, and JIRA for bug-tracking, Confluence for intranet wikis.

But the tester is still alive in me, so here’s my list (all are free):

Rapid Reporter — Shmuel Gershon’s nifty lightweight tool to manage exploratory testing sessions.

PerlClip – James’ nifty little tool to create long data strings for testing.

Expression Encoder 4 — Microsoft’s nifty little tool for recording screencasts (for bug repro or short how-to training videos)

typewith.me — the non-profit Etherpad Foundation’s nifty little tool to (almost) instantly start collaborating on a document or strategy.

corkboard.me — Tim Coulter’s nifty little post-it note creation site (collaboration allowed). 

dxdiag (from the Run command on XP) — meant for info bout DirectX on your Windows PC, but the main screen shows immediate info about onboard BIOS, Memory, Processor, etc.

Ctrl-Shift-Esc — shortcut for bringing up Task Manager in Windows.

Skype, Trillian, Office Communicator — coaching, chat, instant communications

eBay Toolbar — Quick lookup for item IDs and alerting for items I’m watching as part of Live Site testing

FireBug — Firefox (or Chrome) plug-in to quickly find JavaScript errors.

Xobni — quickly searches existing email

FreeMind — for mind-mapping — brainstorming ideas and creates visual “relationships” to other ideas

What’s on your list?

(Maybe I’ll do one on bookmarks next…)

“Winning…”

Recently the Mega Millions jackpot prompted people to play in droves.  News outlets from all over the world ran with the story because it was the largest jackpot in world history.

It amazes me that when people were interviewed about winning, the best they could come up with was “pay bills”, “travel”, “buy a house”, “start a business”…

<yawn>

Really, people?!?   What will you do with the 90% of the jackpot you have left after that?

Let’s say I was the only winner of $640 mil.  That leaves $220 mil after taxes with the cash option.

If I had to keep it testing-centric, here’s what I’d do:

* Buy a Bay Area bookstore, keep it open 24 hours for testers who need books faster than Amazon can deliver. Coffee would be .25 a cup.

* Host a testing conference dedicated to hands-on testing skills and tools — no vendors, sponsors, or fees.  Begin at 10:30 am each day, end when there’s no one left standing. Food and wireless would be available constantly at no charge. Speakers would include people from other disciplines like physics and astronomy — NO motivational speakers.

* Hire my favorite consultants and pay all of their expenses for a year as they help me solve eBay’s top testing challenges.

* Shoot a pilot for a reality show about testing for a start-up company willing to play along.

* Host a nationwide testing competition and broadcast it live as James and I do play-by-play and color commentary.

* Host non-competitive testing *exhibitions* with a mandate that it solve specific problems for people in the audience (chosen well in advance of the show).

* Buy a license of each major tool and post recordings of what they ACTUALLY do with simple testing challenges performed by real testers.

* Start a testing “Smithsonian” with a special section devoted to those in the 1960’s who were trying to solve the same testing problems we have today.

* Create a syndicated daily call-in radio show for testers, hosted by Ben Yaroch.

* Create a testing channel on cable, with shows about testing in science, philosophy, and sociology.

* Hire Ken Burns to do a documentary series about software (with emails from famous testers, read by Morgan Freeman).

Now, what would YOU do with the 90% I would have left after all of that?

CAST-ed

Well, CAST 2011 has shipped.

A few minor bugs, but it looks like the value far outweighed the problems.

James and I wanted it to be THE context-driven event, and it was — speakers from around the world, a tester-themed movie, tester games, a competition, lightning talks, emerging topics, half-day and full-day tutorials, an EdSig with Cem Kaner present on Skype, a live (3D) Weekend Testers Americas session, professional webcasts to the world, debates, panels, open season, facilitation, music, a real-time Skype coaching demo, real-time tweets, live blogs, live LIVE blogs posted as it happened (via http://typewith.me) and red cards for those who needed to speak NOW.

Notice the pattern? Just enough structure to allow play and exploration.  Function and facilities for people to listen and trade ideas.  We provided the place to play and a rough agenda of what could be discussed, but self-organizing professionals did the rest.

And after the conference, they are still organizing.  Sure, I bet a lot of business cards were exchanged, but there’s still traction on the #CAST2011 Twitter hashtag, there are new blogs, there’s talk about EuroCAST and a Brazil CAST.

I may have been presiding officer, but when the show started, I turned it over to the crowd and rode along. Sure, I spoke a few times to the assembled guests, made announcements and reminders, kicked us off in the morning and kicked us out at night, but it was more like being a maitre d’ — I just seated the guests and they made their own dinner conversation.

I’ll help the AST find a place to stage 2012 (likely here in the Bay Area), but I’ll advise next year’s president to make sure there is the same open space to adapt, improvise, and learn so that professionals can decide for themselves if it was worthy of their time.  Lessons here for software development, for sure. 

Until next time, thanks to my core staff — James (theme, tracks and speakers), Ben Yaroch (marketing and media: website, printing, signage broadcasting), and Paul Holland (facilitation, finances, online registration) — to the amazing Danielle Guenther with Lynnwood Convention Center’s extraordinary staff — to Dawn Haynes for turning the reg desk into a full-fledged command center… to the facilitators who kept conversation moving (Paul, Tim Coulter, Sherry Heinze, Nancy Kelln, Mike Goempel).. to Matt Heusser and Peter Walen who encouraged topics to emerge all day, to Michael Larsen for hosting WTA3D, to all of the speakers who sacrified their time, money and sleep… to Doug, Tim, Dee Ann, Cem, Becky, and Lynn for keeping a presence for AST programs and elections, to the sponsors, board members, and people who wanted to help but were turned down because things were going smoothly…

….but last and best, my thanks to the 200 people who showed up in person (and over 800 unique visitors online) who gave it all a reason.

Putting Things in Context

In the movie “The Right Stuff” there’s a scene where character Chuck Yeager is barreling up to the sound barrier in the Bell X-1. 

He’s being shaken to bits from turbulence (transonic speed before 800 mph).  Just when you think the plane will break apart, it breaks the sound barrier (goes supersonic) and all is smooth.

That’s what it’s like to reach the zen of “crazy busy.”  You reach a point where one day you look at your to-do list and everything is Priority 1.  You buffet and shake and are about to implode when you get yet another to-do item to handle.  Then you get another one.  Then, just as you start to fear email, someone walks up to your desk and asks you a question, or worse, reminds you of a commitment you made.  A minute later, you remember you are late for a meeting, and then the phone rings, and then a text message buzzes on your iPhone just as Skype pops up with a request… then, out of nowhere, there is calm. 

It’s a panic-calm as if someone has told you “go to the ocean and put it all into this little cup.”  You can’t do it and you fear failure, but you go to the beach anyway and you see the impossibility of it all and the voice says: “Yes, this was your test. You found your limit. Nice job.”

I am 7 months after my last post here and I’ve been tested and tested and tested and tested and tested.  Sometimes failing, sometimes passing, sometimes the context changes and what used to pass in me now fails, and vice versa.  It’s like being crazy-busy.  You reach a point where you know you are doing the best you can, even though you may forget it 5 minutes later when a new email comes in that tests your abilities.

The tests don’t stop.  And rightly so.  I am made up of a code base to which I am constantly making changes, and with the code that doesn’t change, the Universe seems to provide a test suite of cases with a taxonomy as diverse as James’ Heuristic Test Strategy Model.

The name of Pradeep Soundararajan’s blog is “Tester Tested”.  I’m not sure what he means by it, but I’m assuming he means that to be worth our salt as testers, we have to agree to be scrutinized.

There’s an ocean of things to do as a Quality Director at eBay — as there is with most e-commerce companies.  The site is open 24 x 7, worldwide. At any time there could be a live site issue that affects any type of user Right Now.  I get 200 emails a day.  I am often triple booked and have to choose which meeting I’m going to grace with my presence.  I walk 2.5 miles a day going from building to building talking to people and solving problems.  I come home, spend a few hours with Charlotte, put her to bed, and get right back online — my quiet time to do homework for the next day of classes.

But always always always, there’s me, evaluating myself.  Am I good enough (in this person’s judgment), to accomplish (this task), (at this time)? 

“Good enough” depends on (at least) those three contexts, and I’m embarassed to admit that I have forgotten (more than once) my own advice in Playing the Expert Game.

If you’re looking for those kinds of answers in your testing of software or in yourself, let me say that I’m hoping CAST 2011 starting Monday in Lynnwood, Washington has some answers. It’s a conference dedicated to talking about Context-Driven Testing whether the topic is software or your good-enoughness as a tester.  The speakers there will be tremendous.  They’ll be available and they’ll be willing to be scrutinized unlike most conference speakers.  There will be discussion and debate, competition and collaboration, food and fun.

I’m ready for a bit of a “spring break” from a long semester at eBay to reacquaint me with colleagues I care about and meeting friends I haven’t met yet — those who might have answers I’m looking for.

I tweeted tonight that I have given up my career in testing to learn what I think I *know* about testing. So far, so good.

A colleague tweeted back “Intense is good, otherwise anyone could/would do it.”

That’s a context I was hoping to be reminded of, and I hope not the last for me before it’s time to get back my place by the ocean of all-there-is-to-do.

A man of auction

Wow, what a New Year already…

Opportunities are sometimes fleeting, but this one wouldn’t let me go…

In two weeks, I’ll be in California to serve as a QA Director at eBay San Jose.

Effective January 18, I’ll be leading testers and managers on the “Buyer Experience” team — they’re responsible for all of the tools (mainly search) that help users make  a decision on whether to take part in an auction or to “Buy It Now”.  The quality of the experience of the actual auctions is another department, but if I succeed, I may be able to help those guys, too.

Regardless, in two weeks, consider me an eBay ombudsman.  If you have a problem with an eBay experience, I want to know about it.  It may not be my department, but I’ll funnel it to where it needs to go.

They don’t know yet that I  consider the “A” in my title to be “Assistance”, not  “Assurance”, but they’ll just have to forgive me.

You know me for my ideas, and you know I’ll delight in hiring people smarter than me down there.  Julian Harty is already there and I’ll get to work with him.  So is John Belbute, formerly of Intuit and WebMD. He’s my boss. Above him is VP Todd Paul, ex-senior-Microsoftie and former boss of Harry Robinson on the Bing team.

That’s my new crew. Hard to turn down that kind of team and that kind of brand.  I also have a chance to try some new material and enhance a testing culture.

I thank Quardev for 6 years of opportunities to try my half-baked ideas.  They’re the best testing force in Seattle and they’ve survived ten years with no venture capital, only revenue-funded, privately owned. They’ll do fine without me because the way they approach projects is authentic and honest, and they understand that I need this challenge next in my career.

So, yes, you got that right — one of two uncertified, context-driven Bach brothers will be leading senior test managers at a major brand name company very soon. I’m rarin’ to go, hopeful and optimistic, but also busy repeating the astronaut’s prayer.

I’ll still be shepherding CAST in Seattle, still be going to STAR East and West, STP, QUEST, and PNSQC.  If you’re there, say hi and tell me what you did to find a bug on the eBay site. Hopefully my response will be “I’m on it” or “It’s already in the queue.”

What happens at a real con-FER-ence

Last week, the Software Association of Oregon hosted a one-day Open Space conference.

Accompanied by Andrew Richards, Stephanie El-Hajj, and a cadre of SAO and Portland QASIG / DevSIG volunteers, Diana Larsen did a masterful job facilitating 160 people gathered (at first) in a huge circle of chairs.

An Open Space conference is one where the agenda and speakers are unknown. No one knows what will happen until they get there and start thinking about what they want to talk about. The theme was “Blurring the Line Between QA & Dev in an Agile Environment”, but that was the only structured idea to start with.  People were about to shortly discover where their brain met their heart to figure out what to do next.

Diana asked anyone who wanted to solve a problem to write up a short title on a piece of paper and talk about it for 20 seconds or so in front of the huge circle.  Then they’d tack it up to a huge bulletin board that had time slots and spaces to talk more deeply about it. Then people would decide what most resonated with them.

It just worked.  It met a variety of explicit and implicit requirements to many degrees.

The energy was not too light or too strong. Discussions happened in parallel, allowing people to “vote with their feet” if a topic strayed into malaise for them. At the end, Diana hosted a short retrospective where everyone got a chance to talk (very briefly) about what happened for them.  And everyone did, and it was useful.

It was hard to jump from that wonderful con-FER-ence to one where I thought it would be the opposite.

The next day I was scheduled to speak at the Application Lifecycle Management (ALM) Summit at Microsoft.

ALM Summit Organizer Keith Pleas had approached me back in July saying “We’re targeting the ALM practitioner / lead more than the journeyman developer, and we’re going to be doing sessions more on ‘best practices’ than product features.”

Uh oh.

“Best practices” around what Wikipedia labels “a continuous process of managing the life of an application through governance, development and maintenance… the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management.”

ZZZZZzzzzzzzzzz…..

Knowing that it was going to be held at Microsoft, and that staff from Visual Studio Team System were major players, I did not have high hopes that it was going to be very interesting for me.  It looked like a Microsoft tool-driven conference with preachers preaching to those who already had the tools?  I wasn’t sure.  Just an impression.

I assumed it was going to be as opposite from that Open Space as I could imagine — a dry conference with lots of Information Technology or Enterprise Process Engineering executives wearing three-piece suits.

True to my assumptions, when I walked into building 33 on the Microsoft campus last Thursday, it wasn’t long before I heard many platitudes from speakers:

* “The sooner you fix a bug, the cheaper it is to fix”…

* “Get testing involved earlier in the process”…

* “Communication and management buy-in are very important”…

Attendees (about the same number as that Open Space in Portland) had no choice but to listen.  There was one room, one track, all day… one speaker after another.

But to my surprise, the Twittersphere natives at the ALM conference — the ones using the #almsummit hashtag — were growing restless with this kind of talk.

cacharbe: Ok, we get it, everyone is using Agile. Can we move on to the content?

caffeinatedgeek: @erwilleke I was hoping we would be moving on to the”real world”; HOW and challenges, and not rehashing the WHAT and WHY!

arcs001: After three days of #almsummit we are getting a little bit repetitive…

bsktcase: It’s 2010 and this is ALM Summit. WE don’t need to be sold on agility! We are ready for NEXT steps! Is this what we are saying?

shai_rai: All sessions on Agile Testing at #ALMSummit are using the same Demo and Scenario – Where is The Transparency???

learntfs:  Few are showing what they actually DO on their team. Maybe fearing it looks too much like waterfall? #mindreading

ChadGreen: I love gated checkins and its saved my teams, but how many times do we need to hear about it at

AlexandreLobao: I believe #ALMSummit should have (at least) 2 tracks: 1 for starters, with basics, other focused on real-life problems and advanced topics.

@caffeinatedgeek: feeling like groundhog day at #almsummit. If you only have ONE track, the sessions should be unique.

chrislowndes: #almsummit now turning into TechEd, Season 2, Episode 5 [repeat]

As I monitored the stream of tweets, I saw that the guy next to me had Tweetdeck open, too.  His name was Eric Willeke and not only was he tweeting, he was scheduled to be the next speaker (after lunch).

He tweeted the seed of an idea:

@erwilleke: I’m willing to facilitate roundtables rather than speak today w/ event organizer permission… #almsummit

When he leaned over to tell me his idea, I was sold as soon he had said “What if…?”

All I needed to know was that this guy was reacting to emerging information and was *doing* something novel about it. He was like a developer reacting to results from an exploratory test.  How could I NOT like this guy?!?

Conference organizer Keith Pleas sat to his left.  Eric leaned the other way and whispered to him the idea.

With this, Keith said, “Let me take the temperature of the room about it before lunch.”

And when the current speaker finished (to polite applause), Keith did exactly that.  He went to the front of the auditorium and told everyone Eric’s idea.  It got a rousing ovation.

And with that, the attendees and organizers of the 3-day ALM Summit were about to self-organize. They were responding to change over following a plan — not just words in slides or in a popular Manifesto, but actions.  It was a beautiful, simple, organic thing — no politics, no hesitation, no fear.  Just smart colleagues freeing themselves with a simple idea.

Eric was the right guy to lead this insurrection.  Without dogma or drama, he took the lectern in the large ballroom where lunch was winding down an hour later. He got everyone’s attention and introduced the idea:

“Seekers”, anyone in the audience  who had questions or problems to which they wanted answers, would seek counsel from “consultants”, any volunteer who had something constructive to say about it.  They’d meet at tables and anyone else could sit in and listen. Simple as that.

He timed-boxed it and facilitated the 3 or 4 rules that went along with it.  It was focused and energetic, and it ended right on time — just in time for me to speak next.

D’oh!

I had to follow *that*?!?

I had to follow the rave (and inevitable) success of a self-organizing team with a strong mission and a strong leader?

Uh oh…

Good thing I’m speaking about exploratory testing.  There was no better segue to talk about a testing approach built on the idea of adapting to emerging information, coupled with notions of Agile Development.

I like to think I brought my A-game to my presentation.  I did adapt my talk a bit — punched it up to emphasize (and honor) what had just happened in Eric’s impromptu dojo, but really… what speaker can rank against a group of smart, passionate people who have just replaced a rigid structure that was not meeting expectations?

That, dear friends, is what makes me excited to stage CAST 2011 in Seattle next year.

As James and I plan CAST, we promise it will be built to embrace change. That is our theme, after all — Context-Driven Testing — where Q&A will be facilitated and great ideas will be honored (and tried) if there’s energy around them. If we need to, we will respond to change over following our plan, especially if it means people get to really CONFER at our conference like they did at the SAO’s Open Space and those few adaptive hours at the ALM Summit.

In the meantime, my three heroes this week are Keith Pleas, Eric Willeke, and Diana Larsen.

These three folks embody what it really means to agile and self-organizing, but with grace and simplicity.  All three helped to remind me that the word *confer* is the root of the word “conference”, but also proved why *commune* is the root of the word “community”.

Courting Coincidence (aka “networking”)

I spoke at a panel at Edmonds Community College today.  It was designed to help students get ideas to increase their chances of finding a job.  The other panelists tended to treat it like a resume clinic or job interview seminar.  They missed the point.

The point was *networking* — the ways people connect and relate to each other.

It occurred to me that even if someone in the audience was successful in landing a job, there was no guarantee they were going to keep it.  I was hired once, and laid off 7 months later. The anxiety of the economy might not stop just because you got hired.  Sure, some bills might get paid, but now you have the problem of wondering when they won’t.

It’s tough out there, but networking has saved me. My reputation has been the battle jitney that has not only protected me from some turmoil, but has been conspicuous enough to attract opportunities.  In that spirit, I give you my ideas to increase the likelihood you will be at the right place at the right time or meet the right person with the right opportunity.

* Twitter: tweet something that really resonates with you. Sure, sometimes saying what bagel you had for lunch might help (that’s actually how I met a worthy speaker for QASIG), but give others a chance to know what you find meaningful. Find a hashtag that you’re interested in (like #testing) and reply to someone who said something that resonated with you.

* Facebook: comment on someone’s status with a link you think is relevant and cool. Post your own status about an test idea you have or a thought about a test tool.

* LinkedIn: There are lots of forums in which to participate, so don’t just post a profile and status and leave it there.

*  Blog: start one or comment on one with your full name and email. Give others a chance to say “Where has *this* guy been?!? He’s got game!”

* Seattle Job Social: A regular meetup of employers and seekers in an informal setting — a popular bar in Belltown. I’ve been to two of them — one as an employer, one as a seeker.  Both were useful, but became less so the longer people drank. 🙂

* QASIG: Quality Assistance Special Interest Group — free — meets every second Wednesday of every odd month. Sponsored by Quardev — hosts a speaker and free pizza. (I’m its speaker chairman.)

* SASQAG: The Seattle Area Software Quality Assurance Group — free — meets every third Thursday at Compucom (Excell) – hosts a speaker and free snackies. Keith Stobie and Tracy Monteith from Microsoft are its speaker coordinators.

* SEASPIN: The Seattle Eastside Area Software Process Improvement Network — free — meets every month at Construx — — hosts a speaker and free pizza.  Jeff Smith and Steven Smith (no relation) are its speaker coordinators.

* Seattle Lean Coffee: meets every Wednesday morning at Uptown Espresso — free — hosted by Jim Benson and Jeremy Lightsmith.  Devoted to talking about software development problems in the context of  principles and notions of Lean, Agile, XP, Scrum, kanban.

* Open Coffee: meets every Tuesday morning at Louisa’s Cafe on Eastlake. Devoted to entrepreneurs who want to meet with others, as well as venture capitalists.

* OWASP: The Open Web Application Security Project — a meetup dedicated to discussing software  security issues.  This is a worldwide org.

* Volunteer to be a guest speaker at a community college.

* Your local Chamber of Commerce has an events calendar or newsletter.

* Local Business Journals.  In Seattle, we have the PSBJ with an events calendar.

* Technology Associations. In Washington, we have the WTIA with an events calendar.

* Industry Associations. Like the Association for Software Testing. You can volunteer to be a facilitator or help with the website, or the yearly conference.

* Colleges and tech schools. Volunteer to be on their industry (“real world experienced professional”) curriculum advisory board.

I’ll think of more and post them here.

Comment to add your ideas, too.

Or, do one of these above and tell me how it went.