Ensuring quality on diverse mobile hardware is a daunting task. In the competitive gaming market, there’s just one chance to get it right. There’s some anxiety to launch and complete QA quickly to get your product onto the market and the appstore properly. Here are some useful tips to help you quickly move ahead to launch.
Successful planning will greatly improve your results. The following steps provide guidance on how to plan your test, allowing for some customisation and fine tuning, depending on the game being tested.
Emulators can be extremely helpful to provide a quick turnaround in the early stages of test. If used properly and efficiently, like within an automation structure, they can also allow for massive amounts of data that can increase your QA efficiency drastically. Planning for them, if you can and if applicable, can be a great boon for your development plan.
However, like any tool or practice, there are limitation to what they can allow you to do or verify. So while they can be great, you also need to complement your plan with other tools and practices. Therefore for best results the testing should be carefully planned between emulators and real devices.
“Real-life” devices will allow you to test directly, as well as verify partial results you are receiving from QA emulation and QA automation. This approach will allow you to verify battery drainage, geolocation, network, inbuilt sensors, screen, interrupts or push notification, multiple installed apps interaction, device memory allocation, user behaviour, and more, which is only possible when you have the phone in your hands.
When it comes to devices, it’s important to understand the device selection process before planning QA. Consideration should be given to the device landscape, which is mainly dependent on 3 parameters.
Global Smartphone Market Share- By OS/Platform
Manufacturer: Selecting devices by manufacturer helps to kill two birds with one stone.
Global Ranking of Smartphone Production & Market Share by Vendors, 2017-18
While planning device coverage, select a range of OS, screen sizes, manufacturers, CPU, RAM etc. Stay focused and don’t be overwhelmed by the many manufacturers for Android devices. Search and select the most popular in the targeted region.
Be mindful of the full testing scope and avoid focusing solely on the sales numbers. It’s important to include mid and low end devices to cater to the existing user base.
Global Smartphone Market Share in Q1 2018 – By Devices
Source: https://pricebaba.com, May 2018
Don’t randomly test on each and every device that you can borrow or lay your hands on. This will prevent complete coverage across a range of devices.
The kind of tests you run on your game/app will largely depend on how feature complete your game is. The entire development cycle can be divided into 3-4 stages of completion.
Pre Alpha/Alpha: This is where the game is in a playable state though not all of the assets/features have been implemented.
However as you move towards Beta, it’s important to test graphics and GUI on different screen sizes and resolutions by adding some mid-range devices and tablets (both iOS and Android) to the testbed.
Beta: At the Beta stage, the game is both feature and content complete. A range of tests are required to ensure stable performance such as; Gameplay, Network, In App Purchases, Usability, GUI, compatibility, Performance, Security, Recovery and Interrupts. Correct device selection is critical at this stage.
Devices by Category and Type
Attention should be given to the chipsets, to create some variations in your test bed. For example, Samsung S9+ has an Exynos chipset, so when selecting an Android tablet it would make sense to use a different chipset.
Compatibility Device Matrix- Example
Release Candidate: At this stage, your game is now a stable candidate for release. Once Certification, Interrupts, sanity and regression tests are complete, the game can be launched and submitted to Apple and/or Google.
Devices by Category:
Questions? Do let me know if do things differently.