App testing and The Beta Family

This is the first post in a monthly series of articles on software testing. We intend to cover testing in general, but especially targeted at mobile apps. Developers and testers alike will find useful testing tips and techniques.


Why test mobile apps?
If you were asked to describe the main purpose of software testing you would probably think of finding and fixing bugs and to verify intended functionality and behavior. Quality assurance, simply put. Those are all valid points, but more interestingly, it can help your app to stand out against its competition and ultimately even increase profit.

Have you ever written an app review on Google Play or App Store? If so, how many stars did you rate it? Chances are that you either rated the app with a 1 or a 5. Generally speaking, people cannot be bothered to write a review unless they are emotionally engaged, either raging with hatred or singing the app’s praises. Statistics gathered from almost 200, 000 Android apps show that over 72 % of reviews rated the app either a 1 or a 5. ¹

Think about that for a minute. What would it mean for your app’s average rating if you could eliminate those 1 star reviews? Considering the added impact from a larger share of 5 star reviews, it would make all the difference. And as the app gets featured higher in app store rankings and feature spots, more downloads are likely to follow. Remember, the ultimate goal of testing your app is increased success.

Find the key issues before release
Usually, many of the complaints found in 1­ star reviews would refer to the same limited set of issues. In other words, a handful of problems are probably causing most of the complaints. A distribution of that type is often called 80-­20 or Pareto distributed, indicating that a few factors account for the vast majority of the result. Of course, the challenge is to identify these key issues. ²

The good news is that if app reviewers can find the serious issues with your app, so can beta testers. A beta testing phase using The Beta Family before your actual app release on Google Play or App Store could potentially save you from releasing an app with annoying issues. It will also help you deliver a more high-quality app that’s likely to score higher ratings and attract more downloads.

Common app complaints
Another piece of good news: both Apple and Google want your app to succeed in the marketplace. After all, they get 30 percent commission on your sales. And The Beta Family wants to help out too, even before you publish your app for beta testing. That’s why we’ve researched some of the most common reasons for receiving a negative app review.

The reasons for negative app reviews differ a bit between Android and iOS ³. For instance, Android users complain more about installation difficulties, while iOS users are more negative about app pricing. But taking the operating system aside, Intel’s developer blog lists a few specific common app criticisms ⁴.

  • In-app nagging to follow/like the app or creator
  • Intrusive full-screen ads
  • Non-wrapping text that doesn’t fit the screen
  • Unmotivated or abusive use of Facebook connection
  • Too many emails and push notifications
  • In-app nagging to rate the app on the marketplace
  • Apps that take over too much functionality
  • Freezing/crashing/slow responsiveness
  • Excessive battery drainage

Developers may want to take note not to repeat these mistakes and testers can keep an eye out when writing their test reports. Over the next couple of blog posts we’ll describe in more detail how to test for these types of flaws and how some particular functions of The Beta Family can be of assistance.

³ The state of mobile app quality: Android vs iOS
⁴ Eight reasons Why users hate your app

Published by

Peter Skogsberg

Peter Skogsberg is an ISTQB certified software tester and holds a Master’s degree in Information Technology. He has been on both the development and the testing sides of several mobile and web applications.