Keyword Driven Testing with Ranorex

Experiences, small talk, and other automation gossip.
User avatar
sdaly
Posts: 238
Joined: Mon May 10, 2010 11:04 am
Location: Dundee, Scotland

Keyword Driven Testing with Ranorex

Post by sdaly » Wed Aug 04, 2010 4:13 pm

I have just put together a class for basic keyword testing with Ranorex.

To use it -
Dim test as New KeywordDriver("c:\test.txt")
test.start

Of course you could loop through and execute for a whole folder of keyword test files :)

Within the text file you specify like so -
moduleOrClassName.methodName, param1,param2,param3 etc

You can also leave blank lines and you can use // for comments.


The code is as below -
Imports System.IO
Imports System

Public Class KeywordDriver
	
	Dim keywords as String()
	
	Public Sub New(keywordFile As String)
		If file.Exists(keywordFile) Then
			parseKeywords(keywordFile)
		Else
			Exit Sub
		End If
	End Sub
	
	Public Sub start
		executeKeywords
	End Sub
	
	Private Sub parseKeywords(fileName As String)
		keywords = file.ReadAllLines(fileName)
	End Sub
	
	Private Sub executeKeywords
			For Each keyword As String In keywords
				If trim(mid(keyword,1,2)) <> "//" And trim(keyword) <> "" Then
					Dim strParams As String() = split(keyword,",")
					Dim parameters (strParams.Length-2) As  Object
					For x As Integer = 0 To strParams.Length - 2
						parameters(x) = trim(strParams(x + 1))
					Next
					startMethod(strParams(0),parameters)
				End If
			Next
	End Sub
	
	Public Sub startMethod(methodName As String,  parameters() as object)
			Dim names as String() = split(methodName,".")
			Dim a As system.Reflection.Assembly = system.reflection.Assembly.GetExecutingAssembly
			For Each t as system.Type In a.GetTypes
				If t.Name = names(0) Then
					For Each m As system.Reflection.MethodInfo In t.GetMethods
						If m.Name = names(1) Then
							Dim x As Integer = 0
							For Each paramInfo As System.Reflection.ParameterInfo In m.GetParameters()
								parameters(x) = convert.ChangeType(parameters(x),paramInfo.ParameterType)
								x = x + 1
							Next
							Dim o As New Object
							m.Invoke(o,parameters)
						End if
					Next
				End if
			Next
	End Sub
	
End Class

jagadish2099
Posts: 1
Joined: Thu Jan 20, 2011 12:56 pm

Re: Keyword Driven Testing with Ranorex

Post by jagadish2099 » Thu Jan 20, 2011 6:15 pm

can you please tell what are the contents of the test file: c:\test.txt and how the test case looks like.

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Mon Feb 07, 2011 4:53 am

One might want to look at Robot Framework for Ranorex keyword-driven testing.

Robot Framework itself is keyword-driven and now works with IronPython for .NET support, so it can integrate with or use Ranorex.

However, it might be some amount of work, until some implements a generic Ranorex test library for Robot Framework.

http://www.robotframework.org

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Mon Feb 07, 2011 4:56 am

Nice solution by the way, looks like it could be adapted for other purposes or other test tools like MS UI Test, NUnit?, webAii/Silverlight testing, etc.

User avatar
artur_gadomski
Posts: 207
Joined: Mon Jul 19, 2010 6:55 am
Location: Copenhagen, Denmark
Contact:

Re: Keyword Driven Testing with Ranorex

Post by artur_gadomski » Tue Mar 27, 2012 8:00 am

I have made a small proof of concept project that uses RobotFramework and Iron Python to execute Ranorex code. Reporting needs a bit of ironing out but other than that everything seems to look fine.

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Tue Mar 27, 2012 10:07 pm

artur_gadomski wrote:I have made a small proof of concept project that uses RobotFramework and Iron Python to execute Ranorex code. Reporting needs a bit of ironing out but other than that everything seems to look fine.
That's great artur, I don't suppose you can share the prototype with the public? Would be nice to mention this on the RobotFramework users group as well.

On a side note, I developed a .NET/C# remote library interface (server) for RobotFramework a while back, so those who don't like/want IronPython can build a RobotFramework Ranorex library in pure C# (or VB.NET) w/o IronPython wrapper (or code). The remote server is referenced on the RobotFramework site for where to find and download. There's an example library as well to show you how to interface to it. It uses reflection to load the .NET llibrary, so the source library can be in any .NET language.

User avatar
artur_gadomski
Posts: 207
Joined: Mon Jul 19, 2010 6:55 am
Location: Copenhagen, Denmark
Contact:

Re: Keyword Driven Testing with Ranorex

Post by artur_gadomski » Tue Apr 10, 2012 10:05 am

It's really simple. Each method in C# assembly is a keyword and is wrapped in an IronPython library that is then linked/imported by RobotFramework.

Sample IronPython library:

Code: Select all

import os
import sys
import clr
clr.AddReference("KeywordTest.dll")
import KeywordTest
from elementtree import ElementTree

class LoginLibrary:

    def __init__(self):
        self._prog = KeywordTest.Program()

    def connect(self):
        self._prog.Connect()

    def close(self):
        self._prog.Close()

    def capture_screenshot(self):
        self._prog.CaptureScreenshot()

    def start_log(self, suite):
        self._prog.SetupReport(suite)
        
    def should_login(self):
        self._prog.ShouldLogin()
Sample C# assembly:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
using System.Text;
using Ranorex;
using Ranorex.Core.Reporting;

namespace KeywordTest
{
    public class Program
    {
        public void Connect()
        {
            string path = "/form[@controlname='DialogLoginForm']/container[@controlname='_buttonsPanel']/button[@controlname='btnConnect']";
            Report.Info("Clicking connect button");
            Button b =
                "/form[@controlname='DialogLoginForm']/container[@controlname='_buttonsPanel']/button[@controlname='btnConnect']";
            b.Click();
            Delay.Seconds(11); // TODO: static delay can not be changed bacause connecting might fail.
        }

        public void CaptureScreenshot()
        {
            Report.Screenshot();
        }

        public void SetupReport(string suite)
        {
            TestReport.Setup(ReportLevel.Debug, "Log.rxlog", false);
            TestReport.BeginTestSuite(suite);
            TestReport.BeginTestCase("Case");
            TestReport.BeginTestModule("Module");
        }

        public void StopReport()
        {
            TestReport.EndTestModule();
            TestReport.EndTestCase();
            TestReport.SaveReport();
            
        }

        public void ShouldLogin()
        {
            string path = "/form[@controlname='DialogLoginForm']/container[@controlname='_buttonsPanel']/button[@controlname='btnConnect']";
            Report.Debug("Looking for connect button.");
            Button b;
            bool buttonExists = Host.Local.TryFindSingle(path, new Duration(1000 * 20), out b);
            if (buttonExists)
            {
                throw new Exception("Could not login.");
            }
            Report.Info("Logged in.");  
        }
    }
}

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Thu Apr 12, 2012 7:54 am

Thanks for sharing artur.

Would be great if someone next spent the time to build a generic Robot Framework keyword library that will let you perform all the Ranorex UI operations w/o having to customize their own library for Robot. Something similar in API to Robot Framework's SeleniumLibrary or AutoItLibrary, where behind the scenes we implement/abstract the interfacing of translating the simple Robot API to Ranorex's native .NET method calls. But that's just my wishful thinking...(for promoting both Ranorex and Robot Framework).

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Keyword Driven Testing with Ranorex

Post by omayer » Wed Apr 18, 2012 6:17 pm

It will be very helpful if somebody explain what are these code doing.
Thanks Beginner
Tipu

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Thu Apr 19, 2012 7:22 am

omayer wrote:It will be very helpful if somebody explain what are these code doing.
Thanks Beginner
On a high level, here's a summary:

the code simplifies using Ranorex to write tests using keywords. Ranorex functionality like click some button, type text into field, verify text or state of some field is encapsulated within the keyword library methods. Tests can then be written as plain text in a text file or CSV file / Excel spreadsheet, assuming you have a framework to parse the input file and translate to equivalent keywords to call with the Ranorex keyword library. As in the examples posted, you can also write your own library + framework, or use existing framework like Robot Framework, and build library for it.

there are 2 cases for usage here:

A. Ranorex functions are generalized in the keyword library such that you could write a login test like follows in CSV format:

action,argument1,argument2,...
Click, ButtonIdThatRanorexUses
Type,UsernameFieldId,jdoe
Type,PasswordField,test
Verify text on page,You are logged in successfully.

B. Keyword library written with Ranorex functions grouped into logical units of action such that you call a keyword that performs a sequence of actions rather than single action (click, type, etc.)

keyword,arguments
Login As User,jdoe,test
Go to inbox,
read message,messageId1

where Login As User is a keyword/method in the Ranorex keyword library you write that implements all the login steps, provided you pass in as arguments to the method a username & password. So that when you write test, you don't have to explicitly write all the commands to click and type, the library does that. You just specify test as Login As User followed by arguments of what account to login with.

Makes test writing easier and more maintainable, but does mean you have to separately maintain the "interface" or keyword library, as the low level actions are separated from the test itself.

atom
Posts: 357
Joined: Sun Dec 07, 2008 11:14 pm
Location: Dublin, Ireland

Re: Keyword Driven Testing with Ranorex

Post by atom » Thu Apr 26, 2012 7:19 pm

My advice... stay clear of keyword frameworks...
Instead model your AUT pages/forms into your own classes, with the properties of the classes being the controls on the page/form and the methods being the business actions you can do on the form.
Further you can then serialise your class to xml to capture all data from all controls to an xml file, and vice-vera deserialse to fill the forms controls from an xml file

Test cases then only have to use this class, set properties to fill controls and call methods to do business actions.

To me keyword frameworks are a thing of the past... (when automation had to be "dumbed down" to non programmers)

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Fri Apr 27, 2012 2:42 am

atom wrote:To me keyword frameworks are a thing of the past... (when automation had to be "dumbed down" to non programmers)
Good points atom, I'll keep that in mind for future automation work. But a few points to note:
  • A keyword framework also allows you to include non-programmers into the testing and development process, such as business users, managers, customers. They can help specify/define the tests via the keywords and their arguments. It's like writing a test in English vs .NET/C# class method calls. It can help in the ideal situation, as even QA/testers with programming backgrounds and programmers are not fully aware of all the cases that need to be tested. Customers and business stakeholders know more of the expected functionality than we who develop/test it, because they defined it in the first place.
  • Not all QA and testers, even with "Engineer" in their title (not sure about "software (development) engineer in test" / SDET though), are fully qualified with programming backgrounds as far as I've observed. It is great that we're moving towards SDET, programming minded for test automation, but we're not fully there yet either. Vendors still tout record/playback features for their test tools.

atom
Posts: 357
Joined: Sun Dec 07, 2008 11:14 pm
Location: Dublin, Ireland

Re: Keyword Driven Testing with Ranorex

Post by atom » Fri Apr 27, 2012 11:26 am

Hiya,

True keyword frameworks provide a level of abstraction that can be given to non technical people to write test cases. Most keyword frameworks I have seen have been at the "control level" i.e. put a value in a control, get a value out and validate it, which isnt an abstraction at all, whereas keywords should be at the "business level", doing many gui interactions in one action.

My advice if your going to do this is write you action classes implementing a common interface. You can then develop a small GUI to load all your actions from a DLL, and you can place your action sequence into a DataGrid, and use a PropertyGrid to set the action parameters. Make it all serialisable, and you can save/reload the scenario to xml. Develop a "runner" that you do execute your saved scenarion from the xml file also. This design will mean you can then ask developers to add to your action library by just writing a class implementing your interface!

You should take a look to the new "Behaviour Driven Testing" tools around... this allows non technical people to write their tests in a very simple language, that an automation engineer can then implement. Take a look at Cucumber for example (http://cukes.info/). Also Fitness allows this also to some extent allowing non technical people to write tests as wiki pages, which can then implemented.

As always ensure your non technical people are on board, as a lot of time can be wasted developing your framework only for the manual testers/BA's etc. not to use it.

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Keyword Driven Testing with Ranorex

Post by omayer » Fri Apr 27, 2012 4:06 pm

please provide some demo and walkthrough that was just explained .
Thank you,
Beginner
Tipu

daluu
Posts: 23
Joined: Tue Jun 12, 2007 9:03 pm

Re: Keyword Driven Testing with Ranorex

Post by daluu » Sat Apr 28, 2012 1:28 am

atom wrote:My advice if your going to do this is write you action classes implementing a common interface. You can then develop a small GUI to load all your actions from a DLL, and you can place your action sequence into a DataGrid, and use a PropertyGrid to set the action parameters. Make it all serialisable, and you can save/reload the scenario to xml. Develop a "runner" that you do execute your saved scenarion from the xml file also. This design will mean you can then ask developers to add to your action library by just writing a class implementing your interface!
In today's world, that's partly or totally re-inventing the wheel for a framework/interface when one could look for ready made solution (free/open source or commercial), and focus more on building the "library" classes you are talking about. No problem with that though, more customization for your needs this way.

While on this topic, do you know of any existing free / open source framework that nearly matches what you describe here? I'm talking about the GUI and runner utilities, so that the developers/automation engineers can then just focus on implementing the library classes. I kind of have one, mentioned later, but it's not quite as integrated with .NET as what you describe here, though it supports .NET.
atom wrote:You should take a look to the new "Behaviour Driven Testing" tools around... this allows non technical people to write their tests in a very simple language, that an automation engineer can then implement. Take a look at Cucumber for example (http://cukes.info/). Also Fitness allows this also to some extent allowing non technical people to write tests as wiki pages, which can then implemented.
That's what I was getting at. Using pre-existing TDD, ATDD, BDD tools/frameworks. If you read my earlier posts, I was suggesting one in particular:

RobotFramework.org

it supports BDD, keywords, data-driven testing, all the good stuff. And my most favorite part of it is its cross-platform support (via remote library interface), easy test library (or action/interface class) creation, and good documentation. Granted remove the remote library interface, and the framework is more relegated to Python, Java, and somewhat .NET (via IronPython).

I assessed other frameworks awhile back and even considered re-inventing the wheel and building my own, but came across Robot Framework, and see no reason to do that anymore. I'd rather patch Robot Framework to meet my needs than build a framework or similar tooling from scratch.

The problem of other frameworks like what you mention is that it is dependent on particular platforms like Ruby, .NET, Java. If you need to cover some other platform then you need to integrate with custom coding or find another interface (like different FitNess binding for your platform).

One last thing, building your own framework tooling (GUI, runner, etc.) means you'd have to do it each time at each employer (or use what they already have if that's good enough), as the tooling will usually be kept proprietary. Use open source framework and you can keep the architecture more constant where ever you go, only customizing the library components to meet each of your employer's needs. That is assuming each employer adopts the given open source framework of course.