This post is part of a monthly series of articles on software testing. We intend to cover testing in general, but especially targeted at mobile apps. Developers and testers alike will find useful testing tips and techniques.
We explain the difference between white box and black box testing, followed by some helpful and techniques. Along the way we also define some names and word.
What color is the box?
A common distinction between two vastly different types of software testing, is that between white box and black box. You may associate the latter term with plane crashes, but in this case it denotes that you are testing the software without any inside knowledge about source code, architecture or internal design. This is most likely the case with any app you may test on The Beta Family.
White – the insider
White box testing, on the other and, is mainly used by developers with access to the source code itself. Distinct code units can be separately tested by writing unit tests that assert expected output for given input data. These tests often strive to cover as many if statements and code branches as possible. The measurement of this is called code coverage, often defined as the percentage of code lines covered by test cases.
While the general consensus is that developers theoretically would like 100 % code coverage, it’s also recognized that it’s realistically very hard to achieve. It would require defining the expected results for all combinations of input, variables and states. An advantage however is the re-usability of unit tests, which makes them very suitable for automation. At this point I should also point out that there are other types of white box tests, even those that are based on access to documentation rather than actual source code.
Black – testing the functionality
Let us move the focus back to black box testing, as this is more relevant to tests and testers on The Beta Family. When testing from a black box perspective much revolves around the functionality itself. As Wikipedia defines the technique: “The tester is aware of what the software is supposed to do but is not aware of how it does it”.
So if you’re an app developer publishing your app for testing on The Beta Family, it’s extremely helpful for the tester if you attach some kind of documentation over expected functionality. Without it you’re essentially relying on the tester’s personal opinion of how the app should work. This may be good or bad: it could produce more “bug” reports than necessary, but also give you unbiased first impressions.
Techniques and tricks within black box testing
Depending on the size and complexity of the app you are testing, it may not be viable to evaluate the expected outcome of every possible scenario. Luckily there are a number of techniques and tricks that with high probabilities help to identify possible bugs and errors.
Sanity check – Testers use their common sense to infer correct behavior and output. For example: a calendar event should never end before its start date, a product that is out of stock can not be purchased and a salary should never be negative.
State transition analysis – Given the current state of an object, which actions should be possible to perform? If you are testing a turn-based game and it’s the opponent’s turn, can you still perform your move? If you’re writing an email and have not yet filled in the recipient field, should the Send button be activated?
Equivalence partitioning – Test case input are grouped by similarity to guarantee that every input class is tested at least once. An alarm app may be set to active for weekdays (Monday through Friday) while allowing sleeping in on weekends (Saturday and Sunday). While one certainly could create a total of seven test cases, it is highly probable that two will suffice.
Boundary value analysis – The critical edges of a numeric range are used as input in order to verify the most extreme cases. If we consider a calendar app used to book a certain date, the month number should range between 1 to 12. The smallest and largest valid value should be tested, but also the invalid edge cases. Boundary value analysis could be seen as a special application of equivalence partitioning; as for instance any number higher than 13 likely yields the same result.
Stress testing and Error guessing – Testers use their experience from other software to figure out where errors likely reside and force errors. We briefly discussed this in the end of our previous blog post called How to achieve better testing.
The above are just a few of the testing techniques that exist, but a very good start to help testers get into a systematic and informed approach. More techniques and term definitions are available in our software testing glossary, that’s based on the glossary from the International Software Testing Qualifications Board (ISTQB). We’ll continue to discuss glossary terms in upcoming articles and also providing some tips for those interested in taking the exam to become a certified software tester.
PUBLISHED BY
Peter Skogsberg
Peter Skogsberg is an ISTQB certified software tester and holds a Master’s degree in Information Technology. He has been on both the development and the testing sides of several mobile and web applications.