Testing applications on mobile devices is more challenging than testing web apps on desktop due to
- Different range of mobile devices with different screen sizes and hardware configurations like hard keypad, virtual keypad (touch screen) and trackball etc.
- Wide varieties of mobile devices like HTC, Samsung, Apple and Nokia.
- Different mobile operating systems like Android, Symbian, Windows, Blackberry and IOS.
- Different versions of operation system like iOS 5.x, iOS 6.x, BB5.x, BB6.x etc.
- Different mobile network operators like GSM and CDMA.
- Frequent updates – (like android- 4.2, 4.3, 4.4, iOS-5.x, 6.x) – with each update a new testing cycle is recommended to make sure no application functionality is impacted.
As with any application, Mobile application testing is also very important, as clientele is usually in millions for a certain product – and a product with bugs is never appreciated. It often results in monetary losses, legal issue and irreparable brand image damage.
Basic Difference Between Mobile and Desktop Application Testing:
Few obvious aspects that sets mobile app testing apart from the desktop testing
- On desktop, the application is tested on a central processing unit. On a mobile device, the application is tested on handsets like Samsung, Nokia, Apple and HTC.
- Mobile device screen size is smaller than desktop.
- Mobile devices have less memory than desktop.
- Mobiles use network connections like 2G, 3G, 4G or WIFI where desktop use broadband or dial up connections.
- The automation tool used for desktop application testing might not work on mobile applications.
Types of Mobile App Testing:
To address all the above technical aspects, the following types of testing are performed on Mobile applications.
- Usability Testing– To make sure that the mobile app is easy to use and provides a satisfactory user experience to the customers
- Compatibility testing– Testing of the application in different mobiles devices, browsers, screen sizes and OS versions according to the requirements.
- Interface testing– Testing of menu options, buttons, bookmarks, history, settings, and navigation flow of the application.
- Services testing– Testing the services of the application online and offline.
- Low level resource testing: Testing of memory usage, auto deletion of temporary files, local database growing issues known as low level resource testing.
- Performance Testing– Testing the performance of the application by changing the connection from 2G, 3G to WIFI, sharing the documents, battery consumption, etc.
- Operational testing– Testing of backups and recovery plan if battery goes down, or data loss while upgrading the application from store.
- Installation Tests– Validation of the application by installing /uninstalling it on the devices.
- Security Testing– Testing an application to validate if the information system protects data or not.
Mobile Application Testing Strategy
The Test strategy should make sure that all the quality and performance guidelines are met. A few pointers in this area:
1) Selection of the devices – Analyze the market and choose the devices that are widely used. (This decision mostly relies on the clients. The client or the app builders consider the popularity factor of a certain devices as well as the marketing needs for the application to decide what handsets to use for testing.)
2) Emulators – The use of these is extremely useful in the initial stages of development, as they allow quick and efficient checking of the app. Emulator is a system that runs software from one environment to another environment without changing the software itself. It duplicates the features and work on real system.
Types of Mobile Emulators
- Device Emulator- provided by device manufacturers
- Browser Emulator- simulates mobile browser environments.
- Operating systems Emulator- Apple provides emulators for iPhones, Microsoft for Windows phones and Google Android phones
3) After a satisfactory level of development is complete for the mobile app, you could move to test on the physical devices for a more real life scenarios based testing.
4) Consider cloud computing based testing: Cloud computing is basically running devices on multiple systems or networks via Internet where applications can be tested, updated and managed. For testing purposes, it creates the web based mobile environment on a simulator to access the mobile app.
- Backup and recovery- Cloud computing automatically takes back up of your data from remote location making recovery and restoring of data easy. And also, the storage capacity is unlimited.
- Clouds can be accessed from different devices and anywhere.
- Cloud computing is cost efficient, easy to use, maintain and update.
- Fast and quick deployment.
- Web based interface.
- Can run the same script on several devices in parallel.
- Less control– Since the application runs on remote or third party environment, user has limited control and access over the functions.
- Internet connectivity issues– the setup is on Internet. Network issues affect the availability and functioning
- Security and privacy Issues– Cloud computing is an Internet computing and nothing on Internet is completing secure, so chances of data hacking are more.
5) Automation Vs. Manual testing
- If the application contains new functionality, test it manually.
- If the application requires testing once or twice, do it manually.
- Automate the scripts for regression test cases. If regression tests are repeated, automated testing is perfect for that.
- Automate the scripts for complex scenarios which are time consuming if executed manually.
Two kinds of automation tools are available to test mobile apps:
Object based mobile testing tools– automation by mapping elements on the device screen into objects. This approach is independent of screen size and mainly used for Android devices.
- Eg:- ranorex, jamo solution
Image based mobile testing tools– create automation scripts based on screen coordinates of elements.
- Eg:- Sikuli, Egg Plant, RoutineBot
6) Network configuration is also necessary part of mobile testing. It’s important to validate the application on different networks like 2G, 3G, 4G or WIFI.