HOW TO ANDROID APPLICATIONS TEST?

ANDROID APPLICATIONS TEST

Test Plan

Android testing has several approaches:

• Test Android implementations: it’s the object of CTS

• Test mobile application: it’s the object of this document

To complete these tests, the user can use tools:

• CTS has automated the test plan

This document proposes a test plan manually ran but during the event different solutions to partially automate these tests will be tested

1. Introduction

CTS are designed to test Android implementations.

This test plan is designed to test Android mobile applications. The test cases are intended to be ran manually. But some parts can be automated with specific tools.

1. Pre-Tests

This section provides general tests about launching the application.

2.1. Is the application launched successfully? Yes/No

2.2. Once launched, does the application gain control?

Yes/No

2.3. Does the installation of your application not interfere with the pre-installed applications?

Yes /No

2.4. Can you come back to the application main menu screen in less than 3 seconds?

Yes/No

1. User Interface

This section provides tests about delivery, running the application, display, using and visibility of the application.

3.1. Delivery

3.1.1. Is the apk file provided? Yes/ No

3.1.2. For the client server solutions, is the server software available?

Yes/No

3.2. Running

3.2.1. Does the application provide exit functionality?

Yes/No

3.2.2. If the application is not launched within 6(six) seconds, is a warning message or a progress bar displayed?

Yes/No

3.2.3. If the application is resumed from a suspended state, does it return to this state?

Yes/No

3.2.4. When at the application main menu, is the application suspended?

Yes/No

3.3. Splash screen

3.3.1. If there is a splash screen, is it displayed only during the application launch?

Yes/No

3.3.2. Is the splash screen displayed for long enough to read the text?

Yes/No

3.3.3. Is the splash screen displayed less than three (3) seconds?

Yes/No

3.4. Screens

3.4.1. Does the first screen contain the application name or title in a title bar?

Yes/No

3.4.2. Are the screens properly refreshed after dismissing any another screen?

Yes/No

3.4.3. In the text fields, is the insertion point automatically placed at the end of the field?

Yes/No

3.4.4. Is the delay to display the next screen enough short (e.g. less than 10 seconds)?

Yes/No

3.4.5. If your application displays icons, are they clear, visible and appropriate?

Yes/No

3.4.6. Is the number of screen to browse minimized?

Yes/No

3.4.7. Are each screen displayed for time enough to read information?

Yes/No

3.4.8. Are the screens easily readable? Yes/No

3.4.9. Are the pre filled forms used in order to minimize the amount of writing?

Yes/No

3.4.10. Is the virtual keyboard displayed in all the text fields of the screen when needed?

Yes/No

3.4.11. Is each screen correctly displayed in each mode (landscape, portrait, external screen,) display mode?

Yes/No

3.4.12. Is the application display adapted to the screen size?

Yes/No

3.4.13. During long operation, is an animated visual indication displayed to inform the user?

Yes/No

3.4.14. If the application play sounds, does the application propose an option to mute sound?

Yes/No

3.4.15. If the application play sounds, does the application propose an option to mute sound?

Yes/No

3.4.16. Does the start up screen display the title of the game and allow to access to the functions of the game?

Yes/No

3.4.17. If the game contains user setting, does the application provide a form to change settings?

Yes/No

3.4.18. Does the High scores screen display a list of scores and the names of the players in decreasing Score order??

Yes/No

3.4.19. Does the Help screen give the purpose of the game and instructions about the keys to use?

Yes/No

3.4.20. In the play screen, does the back key allow to back to the start up screen?

Yes/No

3.4.21. When in paused state, is the “Continue” option the first of this list?

Yes/No

3.4.22. In the play screen, when the game is over, does the screen allow to save the score, display the

Total score and notify the game over?

Yes/No

3.4.23. In the optional load screen, does the application list the saved games?

Yes/No

3.5. Sounds, colors & vibrant

3.5.1. Have sounds a specific function?

Yes/No

3.5.2. Does the application use sounds without abuse?

Yes/No

3.5.3. Are colors harmonious?

Yes/No

3.5.4. If sound is used, is it possible to disable it?

Yes/No

3.5.5. If vibrator is used, is it possible to disable it?

Yes/No

3.6. Registration system

3.6.1. If the application use an expiration and/or Registration system, is the notification displayed once after the installation of the application?

Yes/No

3.6.2. Once the application has expired, does the registration message disappear?

Yes/No

3.6.3. If the application has expired, does the expiration message give the option to the user to enter an activation code?

Yes/No

3.6.4. If the application display the number of remaining sessions or days, is it displayed in “About” Dialog or in a specific menu option?

Yes/No

3.7. Visibility

3.7.1. Are images displayed and refreshed properly?

Yes/No

3.7.2. Are the texts entirety (not truncated)?

Yes/No

3.7.3. Is there grammar mistakes?

Yes/No

3.7.4. Does the text can cause misunderstanding?

Yes/No

3.7.5. Is the font of the text readable?

Yes/No

3.7.6. Do selection, scrolling and text highlighting are clear in order to avoid visually confusing?

Yes/No

3.7.7. Have elements an appropriate contrast for readability?

Yes/No

3.7.8. Is the whole screen used?

Yes/No

3.7.9. Does each screen indicate the action to perform into the screen (e.g. “Load”)?

Yes/No

3.7.10. Does the first letter of the first word of a text always start with a capital letter?

Yes/No

1. Interacting with the OS

This section tests the resume functionality and the interaction with the handset.

4.1. Resume

4.1.1. Upon receiving an incoming event (e.g. a phone call or a SMS message), does the application lose control?

Yes/No

4.1.2. If the user has entered data and if the application is going to delete the data without to save, does the application warn the user before?

Yes/No

4.1.3. If the user has entered data and if an incoming phone call or SMS message arrives, does the application save data if the user quits the application to answer to the event?

Yes/No

4.1.4. Is the application able to be suspended, and then resumed in the same screen and state?

Yes/No

4.2. Interferences

4.2.1. If the application updates the user preferences, does it warn the user?

Yes/No

4.2.2. Does the application not write to the handset’s address book without the authorization of the user?

Yes/No

 

4.2.3. Does the application not write to the handset’s calendar without the authorization of the user?

Yes/No

4.2.4. Does the application not change user setting in other applications?

Yes/No

4.2.5. If there is an incoming call and if the application try to send data on the network, does an error message is displayed?

Yes/No

4.2.6. If the application reacts at specific events (like focus change, button click, etc …), is the behavior correct?

Yes/No

4.2.7. If the application supports text speech, is the text speech correctly synthesized in the language of the application?

Yes/No

4.2.8. If the application supports speech recognition, can the application recognize the voice in the language of the application?

Yes/No

4.2.9. If the application uses vibrator, is the vibrator correctly activated?

Yes/No

4.2.10. If the application monitors files, is the access to the files detected by the application?

Yes/No

4.2.11. If the application uses sensors, is the application correctly driven by these sensors?

Yes/No

4.2.12. If the application supports gestures, is the gesture correctly read?

Yes/No

4.2.13. If the application supports gestures and creating gestures, is the gesture correctly reloaded and read after saving?

Yes/No

4.3. SMS

4.3.1. If the application initiates a SIP audio call, does the call succeed?

Yes/No

4.3.2. If the application initiates a SIP audio call, once the call ended, is the link released?

Yes/No

4.3.3. If the application receives SIP audio calls, does the call is correctly received?

Yes/No

4.3.4. If the application receives SIP audio calls, once the call ended, is the link released?

Yes/No

4.3.5. If the application uses SIP audio calls and DTMF tones, do the DTMF tones are recognized?

Yes/No

4.3.6. If the application uses WIFI network, do the operations done by the application are correctly done (scan, initiate and terminate connections)?

Yes/No

4.3.7. If the application communicates with a web server, if the network coverage fails, does the application automatically reinitiate the communication once the coverage is reestablished?

Yes/No

 

4.3.8. If the application communicates with a web server, if the network coverage fails, can the user display the main menu of the mobile phone?

Yes/No

4.3.9. If the application can send a SMS, is the SMS correctly sent?

Yes/No

4.3.10. If the application can send a SMS, is the SMS correctly sent three times after?

Yes/No

4.3.11. If the application can receive a SMS, is the SMS correctly received?

Yes/No

4.3.12. If the application can receive a SMS, is the SMS correctly received three times after?

Yes/No

4.3.13. If the application uses NFC and supports real NFC tags, are data correctly read from a NFC tag?

Yes/No

4.3.14. If the application uses NFC and uses Android devices as NFC tags, are data correctly read from the other Android device simulating a NFC tag?

Yes/No

4.3.15. If the application uses NFC tags, can the application read three times after a NFC tag?

Yes/No

4.3.16. If the application uses auto start application on receiving a SMS, can the application start?

Yes/No

4.3.17. If the application uses auto start application on receiving a SMS, can the application start three times after exit?

Yes/No

4.4. System

4.4.1. If the application supports backup/restore feature, if the user wipes data, reinstalls the application and restore data, the previous data should be available in the application.

Yes/No

4.4.2. If the application is a widget, can it be installed in the home screen application?

Yes/No

4.4.3. If geo location is supported, is the accuracy sufficient?

Yes/No

4.4.4. If geo location is supported, does it save battery and data exchange as soon as possible?

Yes/No

4.4.5. If geo location is supported and if updates are supported, do the location updates work?

Yes/No

4.4.6. If the application shares content with another application, can another application access to these data?

Yes/No

4.4.7. If the application uses web browsing, can it launch the web browser?

Yes/No

4.4.8. If the application read contacts, can it read a phone number?

Yes/No

4.4.9. If the application read contacts, can it read a contact name?

Yes/No

4.4.10. If the application supports calendar and read events, can it read an event calendar?

Yes/No

4.4.11. If the application supports calendar and create events, can it create an event calendar?

Yes/No

4.4.12. If the application read files, can it read a file?

Yes/No

4.4.13. If the application write files, can it write a file?

Yes/No

4.4.14. If the application delete files, can it delete a file?

Yes/No

4.4.15. If an external application is called (as Google Map), is it displayed in less than 5 seconds?

Yes/No

4.4.16. When an event happens and the application exists, does the application exit without disturb the mobile phone?

Yes/No

4.4.17. When an event happens and the application is suspended, does the application run without disturb the mobile phone?

Yes/No

4.4.18. When the Application is suspended then resumed, if a memory card is inserted, does the Application work correctly?

Yes/No

4.4.19. When a memory is inserted, does the application work correctly?

Yes/No

4.4.20. When a memory is removed, does the application work correctly?

Yes/No

4.4.21. When the application is launched, does it correctly run if the device is closed?

Yes/No

1. Multimedia

This section is applicable for applications that support multimedia.

5.1. Protocol

5.1.1. If Face Detector is used, is the face of one person correctly identified?

Yes/No

5.1.2. If the application supports external media files and DRM system, can it use a DRMez media?

Yes/No

5.1.3. If the application support capture image, can it capture one image?

Yes/No

5.1.4. If the application support capture image, can it capture one image two times after?

Yes/No

5.1.5. If the application supports record video, can it record one video?

Yes/No

5.1.6. If the application supports record video, can it record one video two times after?

Yes/No

5.1.7. If the application supports record sound, can it record one sound?

Yes/No

5.1.8. If the application supports record sound, can it record one sound two times after?

Yes/No

5.1.9. If the application allows controlling volume, can the user set the volume at the lowest level volume?

Yes/No

5.1.10. If the application allows controlling volume, can the user set the volume at the maximum level volume?

Yes/No

5.1.11. If the application allows playing DTMF tones, are they accepted for the planed usage?

Yes/No

5.1.12. Can the application display all the supported formats images from the local storage?

Yes/No

5.1.13. Can the application play all the supported formats videos from the local storage?

Yes/No

5.1.14. Can the application play all the supported formats sounds from the local storage?

Yes/No

5.1.15. Can the application display all the supported formats images from the web?

Yes/No

5.1.16. Can the application play all the supported formats videos from the web?

Yes/No

5.1.17. Can the application play all the supported formats sounds from the web?

Yes/No

1. Network

This section is only applicable for applications that support network capabilities.

6.1. Protocol

5.3.1. Does the application meet the protocols implemented on the handset (like HTTP protocol)?

Yes/No

5.3.2. Does the application resolve and establish a connection using an IP address and/or a domain name?

Yes/No

6.2. Communication

5.4.1. Do all data sent by the application are retrieved properly, especially for the maximum amount of data for a message?

Yes/No

5.4.2. Once the application has sent a data and has been terminated, is the port released?

Yes/No

5.4.3. After an error message during data transfer, doest the application properly regain control?

Yes/No

5.4.4. When a data packet is lost, are data correct (i.e. not corrupted)?

Yes/No

5.4.5. Does the application not disturb the network during the test cycle (e.g. obstruction of network traffic,)?

Yes/No

5.4.6. Does the application support delayed network connections?

Yes/No

6.3. Security

5.5.1. If the handset supports encryption, is it able to send secure data?

Yes/No

5.5.2. Are sensible data (like passwords) not stored locally?

Yes/No

5.5.3. Are passwords not displayed when entered into the application?

Yes/No

1. Stress test

This section tests the applications running with low memory availability, interruptions and data management under stress.

6.1. Memory

6.1.1. Does the application show any signs of memory leaks, especially when adding and deleting records?

Yes/No

6.1.2. If there is no more free memory and if the application does not save data, does the application run?

Yes/No

6.1.3. If the storage is full and/or there is no more free memory, when you try to entry new data, does the application display an error message and are data not overwrite?

Yes/No

6.1.4. For network enabled applications, when it receive a data packet larger than the available RAM, does the application not corrupt database?

Yes/No

6.1.5. If the application processes data, when there are almost no more memory free, is the application able to be launched four (4) times? If not, is an error message indicating the application cannot be launched displayed?

Yes/No

6.1.6. For network enabled applications, where there are almost no more memory free, is the application able to send and receive data?

Yes/No

6.2. Data

6.2.1. Does the application appropriately process data packets for a long time?

Yes/No

6.2.2. Does the application correctly save data?

Yes/No

6.2.3. When the handset is suddenly powered off, does the application not corrupt data? Are the last saved data correctly saved?

Yes/No

6.2.4. If the application accepts to add records, is it able to add 30 without crash, cause the handset to reset or corrupt data?

Yes/No

6.2.5. When the application delete data, does it inform the user this operation will be permanent?

Yes/No

6.3. Interruption

6.3.1. During the normal operation, does the application not corrupt data, cause the handset to reset, display an inappropriate error message or do invalid calls/operations?

Yes/No

6.3.2. Does the application resume properly after 8 incoming phone calls and 8 incoming messages? Yes/No

6.4. User interactions

6.4.1. Tap on the screen ten times at different positions, the application should work normally and not freeze.

Yes/No

6.4.2. When the application supports various ways to enter data, are all these solutions run correctly?

Yes/No

6.4.3. Does the application manage correctly multi key press?

Yes/No

1. Documentation

This section tests the documentation of the application.

7.1. Documentation

7.1.1. Are all major functions of the application conform to the features described in the document?

Yes/No

7.1.2. Are all the URL links in the documentation and in the application correct?

Yes/No

7.1.3. Does the documentation indicate the technical support information (phone number, email address and/or web page)?

Yes/No

7.1.4. Is the application provided with an User’s guide in a maximum of 5 (five) PDF or HTML files?

Yes/No

7.1.5. Are Help screens consistent with the functionality of the application?

Yes/No

7.1.6. Are the screen shots consistent with User’s guide?

Yes/No

7.1.7. Does the documentation use the correct terminology as defined in the User’s guide?

Yes/No

7.1.8. Is the user able to perform essential tasks without training or help?

Yes/No

8.2. Minimum requirements

7.2.1. Does the application function properly at this minimum configuration?

Yes/No

7.2.2. Does the documentation include the minimum requirements: memory for the application, supported network carriers?

Yes/No

7.3. Interaction

7.3.1. Advice

7.3.1. If the application supports network capabilities, does the documentation indicate that other services (e.g. phone, SMS messages,) are blocked while the application is sending/receiving data?

Yes/No

1. International Applications

This section is only applicable for international applications.

8.1. Localization

8.1.1. If the language selected on the handset is not supported by the application, does the application default to the correct language specified?

Yes/No

8.1.2. Are the formats formatted appropriately for language’s country: date, time, time zone, week start, numeric separators, phone numbers, address format, name, title, weight, measure and currency?

Yes/No

8.1.3. Do data entry fields accept and properly display international character input?

Yes/No

8.1.4. Are special characters allowed?

Yes/No

8.1.5. Does the application respect the international diversity concerning etiquette, policies and metaphors?

Yes/No

8.2. Translation

8.2.1. In the application, is text content translated appropriately?

Yes/No

8.2.2. In the application, is the text content not truncated?

Yes/No

8.2.3. In the document are formatted values (time, date, number, currency,) properly formatted based on the language’s country?

Yes/No

8.2.4. In the document, are text content and screen shots translated appropriately?

Yes/No