Graphical user-interface elements (i.e. UI-elements) are the visible graphical elements on a computer screen representing stored information within computers. UI-elements do not only represent stored information but also allow users to interact with the software. Common examples for UI-elements are windows, text fields, buttons, labels, lists, selection elements.
With respect to automated software testing, the correct detection and identification of UI-elements of a tested software are important. Thus, the concepts on which Ranorex Studio bases its UI-element detection is a fundamental concept which will be introduced compactly herein.
In this chapter
Concept of UI-element identification
The identification of UI-elements is easy at a first glance. At a first glance because we are used to identifying UI-elements on our own and we know how to use them. But, the underlying concept is rather complex. This section tries to explain the basic principle of UI-element detection.
Let’s have a look at the introduction start page of the Ranorex test application and some of the therein present UI-elements.
Demo application with designated UI-elements
Demo application title
Main menu of demo application
Register panes of demo application
Text label showing a welcome message
Text field for entering a name
Text link resetting the welcome message
Exit button closing the application
User’s point of view
Let us now take the user’s point of view, when working with a software. This will help us understand the challenge in the UI-element identification and therefore gain an understanding of what Ranorex does. Have a look at one particular UI-element, a text input field.
Text input field of demo application
Important to know:
- Why do you know that the red-framed UI-element is a text input field?
Possible answers may be:
- I read it in the user guide of the software
- I know from experience
- Someone told me / showed me
- Because of the prompting ‘Enter your name’ and the ‘Submit’ button
- I guess it is a text input field, but I do not know
In fact, you are not able to look inside of the software to verify that this particular UI-element is really a text input field. And you are not able to look inside of the software to see the definition of its behavior, nor one of its functionalities and its impacts on the software. But with one or more of the before mentioned statements of why you are pretty sure to face a text input field, you created your own ‘inner picture’, or ‘inner representation’ of this UI-element to be a text input field.
Automated UI-element identification
Ranorex tracks, identifies and stores UI-elements in a two-step process which is explained in a simplified form hereinafter.
Ranorex UI-element identification process
UI-element in the GUI if the application under test
Internal representation of UI-elements
Important to know:
- UI-elements in the GUI of the application under test are referenced by a corresponding repository item in the test solution
- Each repository item is given a name and a place within a tree-like data structure which abstracts the GUI-structure of the application under test
- Ranorex identifies the type of the UI-element on a best-guess method and maps it to a set of specific roles and capabilities
- On the other hand, Ranorex creates a specific and unique identifier for every repository item. This identifier reflects the position of the UI-element within the GUI and is known as RanoreXPath
During the analysis of the application under test, Ranorex identifies UI-elements and builds a tree-like data structure based on their position within the GUI and based on their relative position to each other. The below-listed example visualizes this concept:
Example UI-element structure
Building a structure means analyzing the onion-like embedding of UI-elements (left image) within structuring elements in the screen and deriving a proper tree-like data-structure (right image).
Example for UI-element tree
- This UI-element represents the top-level UI-element, the demo application program window
- The UI-element contains nearly all other UI-elements of the demo application (exceptions are context menus, list items, and other UI-elements which are usually organized in separate tree branches
- Examples for UI-elements to be contained within this element are the Exit button, the main menu, or the copyright text label
- This UI-element represents the tab-container with the registers ‘Introduction’, ‘Test database’, ‘Image-based automation’, ‘UI-element test area’ and ‘Upload’ included
- The name of this UI-element is handled over from the underlying application during the analysis of the GUI
- The tab register which is the container for the final text input field is named ‘RxTabIntroduction’ and is visible on the screen when the tab ‘Introduction’ is selected
- This element contains all UI-elements present in this tab register
- The name is handled over from the underlying application
- This UI-element represents the text input field for the name
- Together with the ‘Submit’ button, the text label ‘Enter your name’ and other UI-elements this UI-element is hosted by the upper container
The hierarchical, tree-like structure of identified UI-elements can be seen in the UI-element tree browser view of Ranorex Spy:
Example UI-element tree browser view of Ranorex Spy