GUI TESTING

# Isha Bhandari 1, Sagar Bhandari 2

Abstract This document addresses the Graphical User Interface Testing. The Graphical user interfaces are the doorways to interact with software systems and applications. Though the GUI testing can indeed enhance the safety, usability and robustness of the system, but the testing of GUI is the area which has not seen much development despite of its importance. Of the various available tools for GUI testing, the paper focuses on open source software Selenium. The paper also presents the case scenario to which the tool will be used. The GUI testing an Identity Manager product and the use of the tool for the same will be the major part of this paper. 1. INTRODUCTION

This process involves a lot of labour and the interaction of the test designer with the system is what counts a lot. Also if the test cases are repetitive then the scripts may grow extremely large which again cause the storage problem. [1] SilkTest is the leading tool for automating the functional software testing process of GUI applications. Its powerful test automation capabilities make it the perfect solution for crossplatform, localization, and regression testing across a broad set of application technologies, including Web, Java or .NET and client/server, within the confines of today's short testing cycles. Another tool is Selenium. [2] It is a test tool for web applications. Selenium tests run directly in a browser, just as real users do. And they run in Internet Explorer, Mozilla and Firefox on Windows, Linux, and Macintosh. 3. REGRESSION TESTING [4] Test designers use regression testing to ensure that code modifications do not introduce new errors into already tested code. Regressing the testing process presents special challenges for GUIs. First, both inputs and outputs to a GUI depend on the layout of graphical elements. Changes in the layoutbutton placements, menu organization, and so oncan change the input-output mapping and render older test cases useless. Similarly, the expected outputs used by oracles may become obsolete. Second, developers typically use rapid prototyping for GUIs. This development environment requires efficient regression testing mechanisms that can detect the frequent software modifications and adapt the old test cases to them. 4. TESTING METHODOLOGY

Graphical user interfaces are the doorways to interact with software systems and applications. The GUI responds to user events, such as mouse movements or menu selections, providing a front end to the underlying application code. The GUI interacts with the underlying code through messages or method calls. GUIs make software easy to use, and developers are dedicating a larger portion of code to implementing them. GUIs can constitute as much as 60 percent of an applications total code today. The use of GUIs in safety-critical systems is also growing, making their correct operation imperative. Though the GUI testing can indeed enhance the safety, usability and robustness of the system, but the testing of GUI is the area which has not seen much development despite of its importance. Besides just analyzing the aesthetic appeal of an interface the GUI also mainly addresses on asserting that proper and correct results appear as a result of the action by the user. 2. TOOLS AND TECHNIQUES

[4] The current testing techniques use record-playback techniques. The test designer interacts with the GUI by generating mouse and keyboard events. The tool then records the events and captures the GUI screens in the form of a script. These scripts can be later used to perform the tests again.

Although GUIs differ from conventional software in ways that require unique testing techniques, the overall process for implementing this methodology should follow the steps for conventional software:

Though GUI testing follows different methods and software than the conventional testing but even then the methodology used for conventional software can be applied to it. The steps for the same can be stated as under:1. Define the coverage criteria so as to determine what all have to be tested. And once decided, each user interface event should be tested to determine its correctness. 2. Get the complete set of inputs from the specification and the testing needs to be done using all the inputs in the domain. The various inputs can be from mouse click, fingerprint impressions and the strokes from keyboard. 3. To compare with the results of testing at a later stage the expected outputs should be available and therefore the outputs should be made available in the form of expected messages, snapshots and window positions. 4. The selection for the testing tool to be used should involve various factors such as availability, cost, ease to use, reliability and most important the extent to which it suits the purpose. 5. The last but not the least to interpret and analyse the results. The reports that are generated can be customized depending upon what are the major areas of concern. The failures, their cause, implications and remedies are the ripe products that the GUI testing intends to deliver. 5. COST FACTOR FOR AUTOMATION

Fig. 1: Selenium IDE used in the Mozilla Browser (The figure shows the selenium IDE in the Mozilla Browser. For this the selenium-ide-0.8.6.xpi has to be placed in the Extensions folder under Mozilla installation folder) [2]

The cost incurred for testing software may shoot up to 50-60% of total cost associated with the software. Therefore it becomes very necessary to decide whether to go for automation or not. If the product is in development phase and there is a big probability that the interface can be changed, to automate is not the sensible decision to take. Also there might be the scenario when the deployment over various different platforms causes slight change in the look and appeal. And to make it worse further, sometimes changing the framework over which the application is deployed may render all the previous automation futile. So to trade off at any level is a decision which finds strong dilemma with the various other considerations. 6. SELENIUM

All Selenese test cases are written using a three column HTML table. Each row represents a command or step in the test. The first column of the table is the command name and the second two columns are used for arguments. Because Selenium uses a simple HTML table to format test cases, they are stored in static HTML files that can be rendered by any web browser directly from a local disk. The Selenium IDE is a Firefox extension that captures user actions on a web page and stores those actions as an HTML file formatted in the standard three column layout. Using the IDE, it is easy to generate test cases simply by navigating through an application. The resulting HTML file can be played back to ensure the actions taken when the test was recorded can be performed at a latter time.

A. Introduction to Selenium IDE [2] All Selenium Core tests are written in a simple language called "Selenese". Selenese consists of a linear series of commands each of which may have zero, one or two arguments. Commands are broken down into three groups: actions, assertions and accessors. The Selenium reference guide includes an API reference for all of the built commands. [2] Fig. 2: Selenium IDE with the sample test generated (The figure shows the test that is generated automatically by Selenium IDE. Events captured are recorded in html format)

The following is an example of a simple Selenium test case generated by the IDE. It is nothing more then a basic web page whose only contents are the three column table. This test is saved in a file named "Test Case Name.html". Once a test case is saved by the IDE it can be opened in a standard text editor and additional Selenese commands can be inserted simply by adding a table row.
TABLE 1: HTML FILE FOR A SELENIUM TEST CASE <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>Test Case Name</title> </head> <body> <table cellpadding="1" cellspacing="1" border="1"> <thead> <tr><td rowspan="1" colspan="3">Test Case Name</td></tr> </thead><tbody> <tr> <td>open</td> <td>/idm/someEnvironment</td> <td></td> </tr> <tr> <td>click</td> <td>someObject</td> <td></td> </tr> <tr> <td>clickAndWait</td> <td>someButton</td> <td></td> </tr> <tr> <td>type</td> <td>someTextField</td> <td>someValue</td> </tr> </tbody></table> </body> </html>

corporation, without diminishing security or exposing sensitive information. The management of multiple versions of user identities across multiple applications makes the task even more daunting. To understand the point further let us define a scenario of an Identity Manager product. Typically, products such as Computer Associates' eTrust Single Sign-On Passlogix's Single Sign-On and Blockade Systems' Web Single Sign-On authenticate the user at logon and present the available applications on the desktop. Web access management software such as IBM Tivoli's Policy Director, Computer Associates's SiteMinder, Entegrity's AssureAccess, RSA Security's ClearTrust, Oblix's NetPoint, and Entrust's GetAccess lets administrators centrally control user access, enabling SSO through a policy server that grants authorization rights to each application. D. Problem Scenario Definition The resources which may be distributed are protected. And for the access of the resource the authentication and authorization process has to be done by communication with policy server. The user store which may be LDAP based or Oracle directory or any other stores the credentials and details of the users. The credentials stored in the user store are used to match against those supplied by the used at the time of challenge.
Administrator creates the policies

Policy Server

Users requests resources

Policy Store

B. Tag Library Concepts Although it is very simple to record tests using the IDE, the tests can quickly grow too large to be maintainable. By changing the static HTML file generated by the IDE into a dynamic JSP page, the custom tags can be used to replace chunks of static content. The custom tags can be defined and used in all the scripts thus reducing redundancy. This can also improve the performance also. C. Identity Management With the growth of e-business, organizations are wrestling with the challenge of managing secure access to information and applications scattered across a wide range of internal and external computing systems. Furthermore, they have to provide access to a growing number of users, both inside and outside the

User Store (LDAP, Oracle etc)

Fig.3: Components in authentication & authorisation process

A logical grouping of resources is associated with one or more user stores. In addition, policy logical grouping require one or more administrator accounts. The logical grouping contains resources, rules (allow or deny access to the resources), responses (Responses are actions that are triggered after the rule is fired and they can be selected on the rule fired for example if the user is authenticated but not authorized, if the user is not authenticated etc), and policies (Policies decide how users

interact with the resources). The policies are defined in the policy server which bind together the various objects (rules, responses, user store etc)To check the GUI of such a product various tabs which need to be tested and the paths also have to be covered. E: PROCESS AT A GLANCE

can be added. The scripts can be run in auto mode or presentation mode and the reports can be customized. G. CHALLENGE AHEAD The real challenge and the problem using the Selenium tool in this way is the ever increasing size of scripts. As more and more tests are invoked the scripts increase in size and number. The static data which is present in these html files though solves the purpose but also puts some constraints. For example a test case is to create, view and modify the user store from the user interface. The interfaces that shoot up on after the events are made by the user are tested. There may be testing for certain error messages, message acknowledgments, task completion messages etc. these have to be asserted. And this can be done easily using the selenium IDE. The objects that are created using the script can be used as inputs to test case where the modification of these objects are involved and using the same objects the deletion test case will be invoked. But suppose there is a scenario to test the rule. Rules are associated with some resources. These resources are again present in a logical group which is linked to a user store.

Identity Management Product Test User Interface

Resources Add Delete

Respons Create View Modify Delete

Rule Create View Modify Delete

[3] The process for the entire activity of user requesting a resource till he gets access to it is as follows:1. 2. 3. 4. 5. 6. 7. User attempts to access a protected resource. User is challenged for his credentials and presents them to the web agent or the proxy server. The users credentials are passed to the policy server. The user is authenticated against the appropriate user store. The policy server evaluates is the user has permissions to get access. User profile and entitlement information is passed to the application. The user gets access to the secure application which grants access.

Add Delete

Add response

POLICY
Add resource Delete resource Add rule Delete rule

Delete response

Fig.5: Sample User Interface to be tested.

F. RUNNING SELENIUM TESTS All the objects creation, deletion and modification can be tested using the selenium tool. As already described once the tests are performed manually then they can be saved as html files. These scripts can be run later fro regression testing or otherwise. Worth mentioning that Selenium IDE is not only the recording tool but the scripts can be manually edited and the commands

Now to test a rule, a policy has to be created, resources have to be added to the policy and the user store has to be associated with it and then granting the privileges to the users.
User Store Domain Resource Rule

Fig.6: Flow diagram for objects to be created for testing.

So for testing one object three additional objects have to be created and deleted. This not only makes the script clumsy but also increases load on the server. H. RESOLUTION APPROACH The resolution finds its way through use of tags concept. The tag can be written for creating various objects such as rule, response etc. Such test cases will be saved with .jsp extension. Changing the file name makes the test case easier to access once it is being provided by the application server; it also tells the application server that this file needs additional processing before being served. This also gives the strong feature of java script to be used. Because the custom tags are rendered when the test case is requested, it is possible to control exactly what Selenese commands end up being displayed on the final test. With the static tests recorded by the Selenium IDE the test will always appear the way it did when it was last saved; once the custom tags are used the test may appear differently each time it is accessed. This becomes very important when the environment under test changes: because the output of the custom tags can be re-implemented there is no need to change a large group of tests. The following example shows how the Selenium IDE recorded test case as shown in Table-1 can be simplified using the <test:case> tag.
TABLE 2: HTML FILE FOR A SELENIUM TEST CASE USING TAGS <%@ taglib uri="http://jsp.abc.com/qa/Selenium" prefix="test"%> <test:case name="Test Case Name" env="someEnvironment"> <tr> <td>click</td> <td>someObject</td> <td></td> </tr> <tr> <td>clickAndWait</td> <td>someButton</td> <td></td> </tr> <tr> <td>type</td> <td>someTextField</td> <td>someValue</td> </tr> </test:case>

entire test case has a data loop and the same object can be reused for every loop iteration; objects in the case scope are deleted at the end of the test case. When an object has loop scope it is created and deleted every time it is encountered, this is useful when every loop iteration must have a freshly created object; objects in the loop scope are deleted at the end of each loop iteration. The test suite can be customized i.e. based on the requirements various tests can be grouped together and the results can be taken. If the scope of the objects created by the tags is suite then the same object will be used and this increases the performance. The test cases in .jsp format are put in the application server over which the pages will be deployed. 7. GOING AHEAD Selenium is one of the various options that are available today. So the need of the hour is to look for the other options as well and measure the relative productivity of each before adapting anyone of these. ACKNOWLEDGEMENT This paper would not have seen the light of the day had there not be the support of my colleagues, friends and everyone who has helped in own special way. The help and assistance taken from the unknown sources from internet or elsewhere also finds a place for special acknowledgement. With the hope of proving a help to the IT sector we dedicate this paper to entire IT industry. REFERENCES [1] http://www.borland.com/fr/products/silk/ silktest/index.html [2] http://www.openqa.org/selenium/ [3] http://www3.ca.com/Files/DataSheets/ etrust_siteminder_ds.pdf [4] http://www.cs.umd.edu/~atif/papers/ MemonIEEEComputer2002.pdf

When the test is accessed through the application server, the <test:case> tag will be expanded so that the HTML sent back to the browser looks exactly like the original static test case. Created objects can belong to one of three scopes: suite, case or loop. When an object has suite scope it can be used by multiple test cases in the suite; objects in the suite scope are deleted by the "CleanUp" test. When an object has case scope it is only created once when the test case is run, this is useful when an