Scale up your testing: iOS8 now allows up to 1, 000 beta testers
We’ve mentioned this upcoming feature before and now it’s gone live – public iOS app beta testing for up to a 1, 000 testers, using the new TestFlight integration in iOS 8. The Beta Family is the first service to support and enhance this new workflow, opening up new testing possibilities for developers.
This article will serve as an introduction and a step-by-step guide for setting up your app for public beta testing through your iTunes Connect account and the Beta Family.
For a small recap of the news with iOS 8 and the tighter coupling to TestFlight you can refer to our previous post. There, we described how to set up your iTunes Connect account and prepare your iOS app for internal beta testing. Now, you’re also able to apply the new workflow to your external (public) beta tests.
Test coverage through diversity
While the previous maximum limit was 100 testers, this ten-fold increase allows for much better coverage of devices and locales than ever before. Utilize the skilled and diverse tester community of Beta Family, with more than 14, 000 members worldwide. Make sure your app is compatible with all iDevices and firmwares, while also pinpointing possible localization issues.
Internal and/or external?
As Beta Family already supports the workflow for internal testing processes you now have a choice of how to conduct your testing. The option of internal testing allows for inviting up to 25 people from your organisation using iTunes Connect to distribute beta builds of the app.
Of course, the external (public) testing setup allows for inviting anyone, not just from within your development team, and the limit is, as mentioned, now a whopping 1, 000 people. You can publish up to 10 apps simultaneously, regardless of internal or external testing, and each tester may now use up to 10 devices. Previously this was a 1:1 ratio. So what’s the drawback with public beta tests? Public beta tests require each beta build of the app to adhere to App Store Review Guidelines before starting to test, which might take a while to get approved.
It’s of course always up to the developer to choose their testing setup, but perhaps a suitable combination is to let the app go through one or two rounds of internal testing before releasing it to public opinion. Please let us know in the comments section if you have an interesting testing strategy for your app.
Workflow guide
Apple has released a video tutorial on how to set up your account for testing, you can download it from this link. In addition to Apple’s video guide, we provide some more detailed steps with screenshots below. We assume at this point you have already set up your iTunes Connect account and uploaded a beta build of your iOS app.
1. Login to your iTunes Connect account, where you at the time of writing this will even be greeted by information on the new external testing possibilities.
2. Select the app that you want to open up for external beta testing.
3. Click on the Prerelease tab. As you can see, the testing alternatives are now separated into internal and external, each on a separate sub-tab. Click External in this case.
4. Here you can invite testers from the general public to join the beta test of your app. You can add a tester manually by entering their email address and name, or reuse stored tester details from previous sessions. In this case, we want to add new testers.
5. From this screen you can enter all the details yourself, or import a comma-separated list of tester details (containing all email addresses to invite). This file can be generated directly from Beta Family and we highly recommend using this feature, for simplicity.
Testers will now receive an e-mail with an invite.
When the testers click “Open in TestFlight” they will be redirected to the TestFlight app and asked to install the app,
Testing is now under way and an orange dot will appear next to the beta version of the app.
Conclusion
The process is very much similar to the internal beta testing one – the difference is just the need to get each beta build approved according to the App Store Review Guidelines. So now that you have a 1, 000 beta tester slots to fill out – why not check out the Beta Family community?
PUBLISHED BY
Peter Skogsberg
Peter Skogsberg is an ISTQB certified software tester and holds a Master’s degree in Information Technology. He has been on both the development and the testing sides of several mobile and web applications. View all posts by Peter Skogsberg