What if most of the negative stereotypes associated with software testing were actually those of software checking? In other words, what if most people’s view of testing was completely wrong? Before one could answer those questions, they would first have to know the difference is between testing and checking.
Michael Bolton and James Bach have an idea or two on the difference between testing and checking. In fact, the two co-authored a post on that very subject for the Satisfice blog. I highly encourage you to read their post in its entirety, but I did want to briefly highlight their formal explanation, as I think it will be a subject we come back to in the near future. Here is how they define it:
Testing is the process of evaluating a product by learning about it through experimentation, which includes to some degree: questioning, study, modeling, observation and inference.
Checking is the process of making evaluations by applying algorithmic decision rules to specific observations of a product.
And then, to give a clearer picture, they offer some key distinctions/assumptions:
- Testing encompasses checking, whereas checking cannot encompass testing.
- Checking is a process that can, in principle be performed by a tool instead of a human, whereas testing can only be supported by tools. Nevertheless, tools can be used for much more than checking.
- Testing is an open-ended investigation– think “Sherlock Holmes”– whereas checking is short for “fact checking” and focuses on specific facts and rules related to those facts.
- Checking is not the same as confirming. Checks are often used in a confirmatory way (most typically during regression testing), but we can also imagine them used for disconfirmation or for speculative exploration (i.e. a set of automatically generated checks that randomly stomp through a vast space, looking for anything different).
- One common problem in our industry is that checking is confused with testing. Our purpose here is to reduce that confusion.
- A check is describable; a test might not be (that’s because, unlike a check, a test involves tacit knowledge).
- An assertion, in the Computer Science sense, is a kind of check. But not all checks are assertions, and even in the case of assertions, there may be code before the assertion which is part of the check, but not part of the assertion.
- These definitions are not moral judgments. We’re not saying that checking is an inherently bad thing to do. On the contrary, checking may be very important to do. We are asserting that for checking to be considered good, it must happen in the context of a competent testing process. Checking is a tactic of testing.
As I alluded to in the beginning, I find it interesting to note that many of the negative stereotypes of testing actually apply to what Bolton and Bach define as checking, for instance:
- Testing is a not a boring task. Checking, on the other hand, very well could be.
- Testing is not easy. Checking, on the other hand, is sometimes quite easy.
- Testing cannot really be done by computers alone. Checking can (and probably should) be automated.
- Human testers will never be obsolete, but human checkers (for lack of a better word) might be.
Do you have your own definitions of testing and checking? If so, please share them in the comment section below.
Until next time, happy testing (happy checking, too).