Imagine you’re a software tester who just got a assigned to a project, and you’ve quickly discovered and reported tens, hundreds, thousands of bugs. This makes you an all-star, right?
Now look at it from the other side of things, from the perspective of the company or developer who is receiving those bugs. That mountain of bug reports probably feels like a lot of noise.
So, should testers report every bug they find? It’s a tough question, and one that has sparked an enticing debate within our community of 80,000 testers.
“No, It’s Too Much Noise”
Some testers say no, don’t report every bug you find. These testers claim it’s too much noise for the recipient, thus detracting from the high-priority issues. According to testing expert, Lucas Dargis, “I usually give [new testers] some unusual advice. I tell them to ignore low-value bugs; don’t even bother reporting them”. However, Dargis’ advice isn’t all that unusual.
It is quite common that the development team only wants the most relevant bugs reported. Dargis’ perspective comes from his own experience testing a buggy application and (regrettably) reporting all the bugs:
“I was working on a particularly buggy application. I took the approach of logging every bug I came across, even though I knew most of them were low-value. The bug list grew quickly. Soon I had over 100 bugs, but logging bugs takes time and so I had only been able to test about half the application.
It took our PM a week to weed through all those bugs and in the end, he decided that the majority of the bugs were not going to be fixed. Moreover, I had already used up my allotted testing time so we had to launch the product without full test coverage. As you might expect, our customers found several serious bugs in the app after it was in production.
So while technically I didn’t do the wrong thing (I reported bugs, that’s my job right?), I didn’t do the right thing. I should have focused on serious high-value bugs first and only after we were confident we had found enough of them, move on to the low-value bugs as time allowed. I could have saved myself and the PM countless hours of work, and we could have released a higher quality product.”
Low priority bugs can clutter and overwhelm the development team receiving and attempting to fix the issues. By only reporting bugs in-scope, that need the most immediate attention, dev teams can fix the big issues quickly. It makes a lot of sense.
“Yes, It’s My Duty As a Tester to Report All Defects”
But let’s look at it from the other side of things. A bug, no matter what the value, is a bug. According to testing expert, Ryan LaMontagne, as a tester it is your responsibility to identify and report software issues:
“…I believe it is our job and responsibility to file the defect(s). We as testers don’t know if this will be considered high value or low value, but it will deliver value the client expects.
I always tell my Team to never make an assumption. It’s better to file a defect than let it go. Let the client make the decision on the value unless specified.There is a cost involved with fixing defects later in the development cycle and I feel it’s our job to help the clients by reporting all defects, letting them decide on the value, and save them cost upfront.”
LaMontagne also has a point. It’s hard to determine if the bug is something the development team does – or will – find valuable. A bug that might not be significant now, might be down the road, and the information you reported could turn out to be extremely valuable to the recipient.
Ok, we’ve heard both sides to the debate; so what should testers do?
The Answer: “It depends…”
In the end there is no right answer. It’s a situational question that must be evaluated before each test cycle. As Dargis concludes, “Each cycle or project should be evaluated independently. Taking into account all available information, you need to make the ‘right’ decision for your particular situation.”
By understanding the developer or client’s needs and thoroughly examining the scope of the test cycle, you can determine what bugs to report and if you should report every bug you find. In testing, it’s not about quantity, but about helping developers get smart about their software. Whatever path will best help a company improve the quality of their application is the route the tester should take.
Through utilizing a expert-sourced testing solution that has not only testers – but a project manager who can help triage bugs within the bug-tracking system – companies can get smart about their software and fix the high-priority issues first, all while still maintaining a record of all known issues. And if your team doesn’t want all the low-priority reported bugs, you won’t have to see or deal with them. The best of both worlds.
We want to hear from you! Do you think testers should report every bug they find? Let us know in the comments section.