Last month, the International Society for Software Testing opened its proverbial doors. With more than 30 founding members – including some of the biggest names in testing and many past Testing the Limits guests – the ISST has set itself with the mission of promoting context driven testing and becoming a place for the world’s top testers to share ideas, learn and grow.
Tthis month’s Testing the Limits talks to the four ISST board members to understand why they started the society, what kind of response they received at first, where they hope to see the society go in the future and why context driven testing isn’t just a fad.
Come back tomorrow to read Part II of the interview with Ilari Henrik Aegerter, Iain McCowatt, Johan Jonasson and Henrik Andersson.
******
uTest: Let’s start with the obvious question. Why help start the International Society for Software Testing?
Johan: I meet testers from all over the world who tell me what a profound difference they’ve seen in how they approach testing after having been exposed to the people in our community; how they start to question the so called “best practices” and old factory style testing dogma, as well as changing the way they view testing skills and the importance of thinking skills. It’s clear to me that there’s a need for an organization such as ours, that can help testers in our community come together and actively oppose those who still insist on promoting wasteful practices, meaningless certification schemes and a dehumanized testing approach.
Iain: That casts it in the negative. Whilst reasoned opposition to such things has its place, we also stress the need to convince decision makers that there are viable alternatives.
Ilari: Attempting to change a situation is a better alternative to complaining about how bad things are for testers. Whiners have rarely changed the world. If you believe that current testing doesn’t live up to its potential, if you believe that change is necessary, if you believe that you could gain from interacting with a highly passionate group of testers, then join us and be part of the movement which wants to achieve great things.
Henke: Today there is a huge amount of energy boiling in our community. But it is scattered all over the place and lots of great ideas and initiatives which often never amount to anything. Starting ISST is a way to pool and focus this amazing energy. We can help our community to gather around a few initiatives in a coordinated manner to develop new ideas and by doing so further strengthen our craft. ISST can become the go-to place to find the latest ideas, thoughts and discussions which promote common sense testing.
uTest: What drove each of you to context driven testing?
Henke: Well, like so many others I started out at a time when there was not much of a testing community, and there was little training available for testers. So my team and I figured everything out by ourselves and was mostly focused on getting stuff done. And in retrospect I must say we were pretty damn good at it. But one sunny morning ISTQB Certification came knocking on my door asking me out for a drink. I thought to myself: such a handsome piece of paper with a golden frame must be awesome. So we went out on a date, little did I know that I’d been slipped a mickey. After a couple of years of being strung along by fancy pancy ISTQB and his ugly brother TMap I woke up and managed to escape, but bro, did that hurt! Running naked screaming in the streets I found this friendly group of people that spoke a language I hadn’t heard in years. It was focused on actually doing testing and the skills that were needed to do it well – instead of focusing on templates and drawing arrows to different boxes. Finally I felt at home again, with people who were focusing on the same things that I had in my younger years in the craft.
Iain: My reaction the first time I read about context driven testing was “Meh – that’s just common sense”. It wasn’t until a few years later when I got involved in an outsourced gig and saw the damage done by pretending that testing could be successful using cookie cutter processes in a “test factory” that I realized that what I thought was common sense wasn’t that common at all. I guess I’d always been context driven but only realized it when I experienced the alternative.
Ilari: Much of the software testing domain today is occupied by either bureaucrats shuffling piles of paper or technocrats, who think that a machine will solve all their problems automatically.
Iain: I like that, I never quite thought of it that way.
Ilari: …It helps to understand that software is built by humans who make mistakes and that the users of software are also human. Having worked in both the medical domain and in e-commerce, it is obvious to me that there cannot be one single approach to testing. Of course, I was also heavily influenced by some people in the context-driven community.
Johan: I started out as a hard core factory style tester. I was taught to execute test scripts precisely as written so that you can “prove” what’s been tested later on, that bugs not found by a test case aren’t real bugs and that the pass/fail rate is the true measurement of quality, that whole schtick. My first real contact with context-driven testing was in 2007 when I took the Rapid Software Testing course with James Bach. That opened my eyes to how inefficient my testing approach was at the time and it became the trigger for me to start looking beyond the ISTQB and TMap books in my bookshelf and go out and join up with the context-driven community to learn more.
uTest: When working with young testers, what’s the most difficult lesson to impart?
Johan: I think that one of the more challenging things for testers, both young and old, is to be able to explain their approach to testing in a succinct and convincing way. Simple questions that trigger complex answers, like: What’s your plan? Why is that a bug? How will you test this? As a tester you probably “feel” the answers to all these questions, but can you verbalize them? Can you convince somebody who does not live inside your head? That’s challenging to many people. It’s a different skill master and requires a lot of practice and judgement.
Ilari:I don’t know what the most difficult lesson is, but I think a difficult lesson to impart is that there are limits to whatever mental models you have of testing. I find it rather challenging to convey the possibility of the unknown unknowns beyond these mental models.
uTest: The ISST isn’t a giant fan of formal certifications. Why? What do you propose in its place?
Iain: I’m a former member of the Canadian Software Testing Board, a member board of the ISTQB. I’m heavily certified, and personally feel that I benefitted from the education I gave myself – over the course of a few years – in pursuit of certifications. I believe that is possible, but have come to question the value of short course and multiple choice exams. I’ve worked with a lot of testers who went that route and didn’t appear to get much out of it. So whilst I’d say that those I worked with at the CSTB genuinely seemed to care about improving testing, I’d prefer to focus my energies on trying to change the way people think.
Johan: I wouldn’t say I’m against certifications in principle. However, I have yet to find a testing certification scheme that I think is rigorous and diverse enough in the skills they require people to demonstrate to pass certification. I also believe that if you are willing to certify somebody, then there needs to be some sort of accountability attached to the certifying body. Maybe not in the legal sense, but as an employer I should be able to trust that the holder of a foundation certification in software testing have at least a modicum of actual, practical testing skill, not just memorization skill. There certainly aren’t any certifications out there today that let me do that, and so I have to ask: Who is really benefiting from today’s certification schemes? Certainly not testers or their employers.
Ilari: To exemplify the ridiculousness of the testing certifications, imagine a certification scheme for, let’s say, Java developers. They get handed out a multiple choice questionnaire with questions like: “What is a class?”. They cross 40 boxes and get certified. Now, what sane person thinks that after such an exercise anybody would have mastered programming? Or try the same exercise with medical doctors. Pure quackery. Skill is not built by memorizing “correct” answers, but by a strenuous cycle of doing and failing until becoming a master.
uTest: As the board members of ISST, what do you hope to accomplish?
Johan: I hope to be able to give the ISST a good launch and help support the initiatives our members feels most passionately about. Whatever we can do to help testers come together more easily at workshops, webinars, conferences and the like would be fabulous to see and support. The board’s job is to facilitate the need of its members.
Iain: And to provide leadership. We’re not facilitating a social club for testers – yes we see community as an important means of achieving our mission – but we also need to be setting the tone and direction, and providing an environment in which our members energies can be focused on the mission.
Ilari: Change the world as we know it! If the ISST becomes a reference for people seeking skillful testing and is able to influence decision makers, then we all have done our job well. If its members feel pride to be a member of the ISST, then we have not failed. I hope to accomplish a change in thinking about testing with as many people as possible.
Henke: Also that the ISST becomes, through its actions and those of its members, a respected counterweight to the massive machinery that sells fake testing. That testers, when asked what testing credentials they hold, can reply that they are members of ISST and through this be far more credible than through holding any certification.